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Two Emergency School Aid Act (ESSA) --funded 
compensatory education reading programs served 1900 Minneapolis 
students in desegregated school3 during 197*1-75. Both programs 
generally met their objectives for comprehension gain among disabled 
readers. Students in the ESAA Intermediate Reading Program achieved a 
median rate of about 3 grade-score months of comprehension gain for 
every month enrolled in the program. Students in the Junior High 
Reading Program made slightly less than 2 months gain per month 
enrolled if such students entered the program with pretest grade 
scores of 3*9 or less. Por Junior High Program students entering with 
grade scores of 4.0-€»0, the median monthly gain rate was about 3. 
Differences among schools in gain rates are discussed in this report. 
Reading gains were measured using Gates-HacGinitie Primary C or 
Survey D comprehension tests. Both programs emphasized the use of 
audiovisual teaching machines, and commercial and 

Hinneapolis-Schools-produced lessons usable with these machines. The 
frequency of use of various materials is reported. An evaluation of 
these programs was conducted by the Minneapolis Schools" Research and 
Evaluation Department. , In the event that programs like these are 
funded in the future, the evaluator recommends (a) changes in 
pre-post testing procedure, including alternate forms and 
diagnostictype tests; (b) use of a control-group evaluation design; 
(c) careful consideration of information needs among staff and 
funding agencies before beginning evaluation; (d) greater efforts to 
recruit both Native Americans and teachers with reading certification 
for staff positions. (Author/BC) 
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The Intermediate ax^ Junior High 
Reading Programs of the 1974-7S 
Minneapolis Emergency School 
Aid Act Project: 

An Evaluation 

Summary 

See Pages 

Two ESAA-funded compensatory-education reading prograins served 1-2 
1900 Minneapolis students in desegregated schools during 1974-75. Both 
programs generally met their objectives for comprehension gain among 27-30 
disabled readers. Students in the ESAA Intermediate Reading Program 27 
achieved a median rate o£ about 5 grade-score months o£ comprehension 
gain for every month enrolled in the Program. Students in the Junior 
High Reading Program made slightly less than 2 months gain per month 29 
enrolled if such students entered the Program with pretest grade scores 
of 3.9 or less. For Junior High Program students entering with grade 29 
scores of 4.0-6,0, the median monthly gain rate was about 3. Differ- 
ences among schools in gain rates are discussed in this report. Reading 30-35 
gains were measured using Gates-MacGinitie Primary C or Survey D 5-7 
comprehension tests. 

Both programs emphasized the use of audiovisual teaching machines, 8-10 
and commercial and Minneapolis-Schools-produced lessons usable with 
these machines. The frequency of use of various materials is reported. 10-14 

The Intermediate Program, operating in 18 public elementary 
schools, had a staff of 15 teachers,^ 3 part-time tutors, and 16 
teacher aides. The Junior High Program operated in 8 public junior 
highs and 5 nonpublic schools with junior-high-level grades. The 1-2, 18 

combined public and nonpublic Junior High Program had 15 teachers 
(8 full-time teachers in public schools; 7 part-time teachers in non- 
public) and 17 teacher aides. Each program was coordinated by a read- 
ing resource teacher. 

The total ESAA budget for reading was $595, 8S0, representing 48% 2 
of the entire $1,247,256 in ESAA funds awarded to Minneapolis for 1974-75. 
For each of the ESAA reading programs, the amounts allotted per student 
were $331 for the Intermediate Program, $309 for the Junior High Program 15 
in public schools, and $223 for the nonpublic part of the Junior 
High Program. 

An evaluation o£ these programs, based on a research design specified 
in the Project application, was conducted by the Minneapolis Schools' 2 
Research and Evaluation Department. 

r^In the event that programs like these are funded in the future, 
the evaluator recommends (a) changes in pre-post testing procedure, 
including alternate forms and diagnostic-type tests; (b) use of a 36-40 
control-group evaluation design; (c) careful consideration of information 
needs among staff and funding agencies before beginning evaluation; 
(d) greater efforts to recruit both Native Americans and teachers with 
reading certification for staff positions. 

* * * 

May 1976 Research and Evaluation Department 
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The Intermediate and Junior High 
Reading Programs of the 1974«7S 
Minneapolis Emergency School 
Aid Act Project: 

An Evaluation 

During 1974-7S the Minneapolis Schools were awarded $595,850 under 
the provisions of the Emergency School Aid Act (ESAA) to operate two read- 
ing programs. The ESAA Intermediate Reading Program served 989 students 
in grades 4-6 in 18 public elementary schools. The ESAA Junior High Reading 
Program served 911 students in grades 7-9 in 8 public junior highs (758 
students) and in 5 nonpublic schools (153 students). 

The Emergency School Aid Act (1972) has been the federal government's 
primary effort to help solve the instructional and human relations problems 
of school districts undergoing planned desegregation. Accordingly, the 
Minneapolis ESAA reading programs were placed in public elementary and junior 
high schools desegregated as of fall, 1974. (Elementary schools were deseg- 
regated in fall, 1^74; junior highs had been desegregated in fall, 1973. 
See Minneapolis Public Schools' desegregation-plan summary referenced 
on p, 41 of this report.) 

Both Minneapolis ESAA reading programs were similar in their objectives, 
materials, methods, and organization. Both programs sought to improve reading 
comprehension among students wirh poor reading skills. Both provided the 
same teaching machines and many of the same machine-usable reading materials. 
Both employed reading teachers and teacher aides to work in participating 
schools. The same pre-service and in-service training sessions were attended 
by teachers and aides of both programs. Each program was coordinated by a 
full-time lead reading resource teacher, who helped teachers and aides 
implement the program in their respective schools. 

Schools participating in the 1974-75 ESAA readi ng programs. Grades 4-6 
in the following 18 elementary schools participated in the Intermediate 
Reading Component: Anwatin, Bancroft, Bremer, Bryn Mawi-, Clinton, Corcoran, 
Field, Greeley, Hawthorne, Holland, Irving, Lincoln Intermediate Center, 
Lowell, Madison, Northrop, Shingle Creek, Webster, and Whittier.'^ 

The Junior High Reading Program served students in the following 8 public 
junior highs: Anthony, Bryant, Franklin, Jefferson, Jordan, Olson, Phillips, 
and Ramsey. The Junior High Program also served nonpublic school students 
as follows: 7th and 8th graders at Ascension, Holy Rosary'^, Incarnation, 

^The Junior High Prograjn at Holy Rosary also served eight 6th grade students. 
These students were considered part of the Junior High Program for data 
analysis purposes. 
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and St. Stephen; and 9th graders at Regina High. 

Background: The Minneapolis Schools^ ESAA Project 

The Intermediate and Junior High reading programs constituted about 
half of the entire 1974-75 Minneapolis Schools' ESAA Project. As described 
in the Project's application for funding (Emergency School Aid Act Office, 
Minneapolis Public Schools, 1974), the Minneapolis ESA.^ Project had seven com- 
ponents and a total awarded budget of $1,247,256. Of this total, $327,218 (26%) 
was allotted for the Intermediate Reading Program component. Another 
$234,331 (19%) was allotted for reading in public junior highs; and $34,101 
(3%), for grade 7-9 reading in nonpublic schools. While budgeted in the 
Project application as separate components, public and nonpublic junior 
high level reading were identical in their objectives and organization, 
and had the same lead reading resource teacher. The public and nonpublic 
junior high level reading components are therefore considered in this 
report to be a single Junior High Reading Program. 

The fourth and fifth components of the 1974-75 Minneapolis ESAA Project 
provided Desegregation Counselor Aitles for public elementary and secondary 
schools, respectively. These ESAA Counselor Aides worked to prevent or 
resolve various interpersonal conflicts that might arise in their desegregated 
schools. 

The sixth component provided math aides for secondary schools. 

'iJie seventh component, administration, also included $16,000 budget 
for project evaluation. The evaluation of selected components of the ESAA 
Project was conducted by a research psychologist on the staff of the 
Minneapolis Schools' Research and Evaluation Department. 

The development and partial demise of the Minneapolis ESAA Project , 
1973-1975 . The ESAA Project began in Minneapolis Schools in fall, 1973. 
The development of this project paralleled the implementation of the Minneapolis 
desegregation plan. During 1973-74 Minneapolis received $535,441 in ESAA 
funds for reading, math, and counselor aide programs in the newly desegregated 
junior highs (and in the corresponding gi^ades of selected nonpublic schools 
serving multiracial student bodies) . The 1973-74 junior high counselor aide, 
math, and reading programs were described and evaluated by the present 
evaluator (Higgins, 1974a, 1974b, and 1974c). 

In fall, 1974, the planned desegregation of Minneapolis elementary 
schools was carried- out. Simultaneously, new FSAA-fundcd Intermediate 



Reading and Elementary Desegregation Counselor Aide progrrims were implement- 
ed in these desegregating elementary schools. 

At the same time that the ESAA Project was extended to elementary 
schools, the Minneapolis ESAA math program was sharply curtailed. During 

1973- 74 the ESAA Mathematics Component served 1200 junior high students 
with innovative materials and 25 teacher aides. In 1974-75 the ESAA Project 
funded only 9 teacher aides for math and no math materials. 

The Minneapolis Schools sought to maintain and extend its ESAA 
Project for 1975-76, the third year of operation. In July, 1975, however, 
proposed reading and math programs for elementary and secondary students 
were rejected for funding by the federal ESAA administration. 

The Minneapolis ESAA Project continues in 1975-76 with only the 
elementary and secondary Desegregation Counselor Aides Program relatively 
intact. 

The relationship between ESAA and Title I assistance. Most of the 
schools participating in the ESAA reading programs also received federal 
aid under Title I of the Elementary and Secondary Education Act. Duriiig 

1974- 75 all of the ESAA reading program schoo?r> except Field, Shingle Creek, 
Anthony, Olson, Ramsey, and Regina were also Title I schools. 

Title I funds are used to aid the compensatory education of disadvan- 
taged students. ESAA funds, to aid desegregating school districts, are not 
necessarily sarmarked for compensatory education, although they may be so 
used. In Minneapolis, the ESAA Project from 1973 to 1975 did, however, 
emphasize compensatory reading instruction, ESAA students in a Title I 
school were presumably eligible for Title I assistance if they were, as 
expected, among the school's poorest achieving readers. If a school had 
both a Title I reading program and an ESAA reading program, a given ESAA 
student might have been served by both programs. The extent to which ESAA 
students were in fact served by Title I reading programs has not been 
determined. Any such duplication of f,ervices was probably most rare at 
the junior high level, where Title I programs were least well funded. 

The Organization of This Evaluation Report 

Tlie following sections of this report provide answers to four questions 
about the 1974-75 ESAA Intermediate and Junior High reading programs: 

1, What were the objectives of each program? 

2, How did each program operate? The answer to this question includes 
description of each program's curriculum, students, staff, and day- 
to-day operation. 

10 



3. Was each program effective in meeting its objectives? Included in 
the answer to this question is a discussion of factors possibly- 
related to differences among students and schools in reading gain 
rate. 

4. What recommendations should be made concerning the improvement of 
these reading programs, if the same or similar programs are again 
implemented in Minneapolis? 

Sources of information regarding the ESAA Reading programs , htost of 
the information used to prepare this evaluation report was obtained from 
paper-and-pencil instruments completed by the ESAA teachers and aides in 
each school. For each student in the Intermediate and Junior High programs, 
and teacher and/or aides filled out an appropriate Student I nformation Form. 
The Intermediate form is reproduced herein as Appendix A; ^ Junior High 
form, as Appendix i. ESAA teachers and aides also answered questionnaires 
describing their own job, their background, the ESAA reading program in 
their school, and their suggestions for improvement of the ESAA program. 
The ESAA Reading Teacher Questionnaire and the ESAA Reading Aide Question- 
naire are reproduced as Appendices C and D. For a number of items on each 
questionnaire, the responses of the total teacher or aide group have been 
tabulated on the sample form. 
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THE OBJECTIVES OF THE ESAA READING PROGRAMS 



Both reading programs were designed to improve reading comprehension 
•among poor readers in each participating school. 

The Objectives of the Intermediate Reading Program 

The Project application (ESAA Office, Minneapolis Public Schools, 
pp- 53-54) expected "ESAA- eligible" students in grades 4-6 to show the 
following gains in reading comprehension when they actually received ESAA- 
funded reading instruction: 

1. "25% will make 1,4 [or more] months gain for each month in the 
program." 

2, "25% will make 1.0-1.3 months gain for each month in the program." 

5. "The median gain for all students served in the program will be l.O 
times "$he; number of months in the program." 

An implied objective therefore was that 50% or fewer students would 
make less than 1,0 months gain for each month in the program. 

Eligibility for instruction . On p. 53 of the' Project application, 
eligibility for ESAA instruction was defined in terms of the discrepancy 
between a student's actual grad^ placement and that student's grade score 
on a reading comprehension test. Fourth graders needed to be at least 1.5 
grade- score years below actual grade placement; 5th and 6th graders needed 
to be at least 2 years below grade level in their reading achievement. 
These criteria for ESAA eligibility proved unworkable in practice, however, 
generating too few students at each school. The criteria for eligibility 
were therefore changed early in 1974-75 to include any student o ne or more 
years be] " ^w grade in reading . 

Testing. Reading gain was to be measured using "the appropriate form 
of the Gates-MacGinitie Comprehension Test" as both pretest and posttest 
[application for funds, p. 54). The actual comprehension tests used were 
from either Primary C, Form 2, or Primary C, Form 1, of the Gates-MacGinitie 
Reading Tests. Both tests, supposedly having the same difficulty level, 
were originally designed for use with 3rd grade students. 

Most of the Intermediate Program students took the Comprehension Test 

of Primary C, Form 2 (hereafter abbreviated as C2) for both pretest and 

posttest. The evaluator had, however, in a previous year's ESAA evaluation 

(Higgins, 1974c, pp. 21, 26) recommended the use of alternate, equivalent 

forms for the testing of reading gain. The staff of the Intermediate 
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Reading Program partially implemented this recommendation in spring, 1975, 
by using the CI as a posttest for some students in some schools. 

The Objectives of the Junior High Reading Program 

The Junior High Reading Program was aimed at the poorest-reading 
students in each ESAA school. In each ESAA public junior high, the lowest- 
achieving 125 "readers*' were the target of the Reading Program. In the 
nonpublic ESAA schools, smaller numbers of poor readers were selected for 
the program. Each student selected for the ESAA Reading Program was 
supposed to be two or more reading-comprehension years below actual grade 
placement, at the start of the 1974-75 school year. Also, no student 
was to be above the 6.0 grade level in pretest reading comprehension. 

The Junior High Program divided students into two groups, depending 
On their pretest comprehension grade score, and then assigned a different 
set of reading gain objectives to each group (see Project application, 
pp. 17-18): 

Objective Set I : students entering the Junior High Program with reading- 
comprehension grade-equivalent scores of 3.9 or less were designated 
Objective I students and were expected to show the following gains in 
reading comprehension: 

1. *'30% will make 2.5 [or more] months gain for each month in the 
program." 

2. *45% will make 1,7 to 2.4 months gain for each month in the program." 

3. "30% will make 1.0 to 1.6 months gain for each month in the program." 

4. "The median gain for all students enrolled in the program will be 
1.5 times the normal number of months in the program." The 
evaluator interpreted the fourth objective to mean that Objective I 
students would gain 1,5 or more grade-equivalent-score months for 
every calendar month in the program. 

Objective Set II : students entering the Junior High Program with reading- 
comprehension grade-equivalent scores of 4.0 to 6.0, but at least two 
achievement years below their actual grade placement, were designated 
Objective II students and were expected to show the following gains in 
reading comprehension: 

1. "30% will make 4.0 tor more] months gain for each month in the program.* 

2. "15% will make 2.7 to 3.9 months gain for each month in the program." 

. 3. "30% will make 1.0 to 2.6 months gain for each month in the program." 

4. "The median gain for all students involved in the program will be 
2.5 times the normal number of months in the program." Again, the 
evaluator interpreted this objective to mean that Objective II students 
would gain 2.5 or more grade-equivalent-score months for every calendar 
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month in the program. 

Eligibility for instruction . The application for funds stated that 
the Junior High Reading- Program would serve "students in each school who 
are 2 or more years below grade level in reading (p. 14) The fact 
that objectives were not stated for junior high students with grade scores 
greater than 6.0 implied that such students were not eligible for the 
Junior High Reading Program. 

Testing , The measurement of J ram reading objectives 

was conducted using two tests. Foi w students (pretest grade- 

score reading comprehension level of 0-3.9) reading gains were measured 
using the same Gates-MacGinitie Primary C, Form 2 Comprehension Test 
used in the Intermediate Program. For Objective II students (4.0-6.0 
pretest comprehension level) the Gates-MacGinitie Survey D, Form 2 or 
Form 2M, Comprehension Test was used for both pretest and posttest. The 
D2 and D2M tests had the same items but differed in the way students 
recorded their answers. Using D2, students wrote their answers on the 
test booklet; using D2M, students wrote on a separate machine-scorable 
answer sheet. 

Unlike the Intermediate Program, in no case did Junior High Program 
students use alternate, equivalent forms for pretest and posttest. 
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THE OPERATION OF THE ESAA READING PROGRAMS 



This discussion includes a description of each program's instructional 
approaches; a description of the students and their method of selection for 
each program; and a description of each program's day to day administration 
within the school. 

Instructional Approaches Common to Both Reading Programs 

Instruction in both the Intermediate and Junior High reading programs 
emphasized the use of innovative curriculum hiixicr' able with the 

Dorsett M-86 A-V Teaching Machine. The Dorsett mr^^hine, resembling a small 
television set, contains a filmstrip projector and a record player. The 
Dorsett Company also added a cassette player to each machine, to accommodate 
lessons from the Basic Skills Centers Reading Program (see below). ESAA 
funds were used to place Dorsett machines in every elementary, junior high, 
and parochial school participating in either the Intermediate or Junior 
High reading programs. The only exceptions were the three intermediate 
schools that had half-time reading tutors; machines and machine-usable 
materials were not provided to these schools. 

Each lesson for an audiovisual teaching machine such as the Dorsett 
consists of both a filmstrip and a synchronized soundtrack. For each frame 
of the filmstrip there is a soundtrack-presented comment or question. If 
a question is asked, the student responds by pressing one of three buttons 
(some audiovisual machines have five buttons) . A correct choice is follow- 
ed by a soundtrack presentation of ''Yes," '^Right," "Correct," or the 
equivalent, and the filmstrip automatically advances. On the Dorsett 
machine, an incorrect choice is followed by a 1-second "error tone," and 
the correct button must then be pressed for the lesson to continue. At 
the conclusion of the audiovisual presentation, the student may complete 
a brief paper-and-pencil mastery test. Each lesson used in the reading 
programs generally took 15-20 minutes; most students could easily complete 
two lessons during a class period. 

Two sets of curriculum materials were used with Dorsett machines in 
each reading program. The previous comments on the design and use of the 
materials generally apply to both of the following curricula: 

1. The Basic Skill Centers Reading Program , developed by Minneapolis 
Schools staff, was first used in the Basic Skill Centers of the Minneapolis 
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Public Schools (see Clark, 1972, 1973). Each lesson in the Basic Skill 
curriculum focuses on a particular ^'molecular** reading skill, or several 
related skills. With nonreaders, the lessons are used in an invariant 
sequence to develop systematically the primary phonetic decoding skills 
of word analysis. Individual lessons in the Basic Skill curriculum can 
also be used in remedial work with readers having specific weaknesses. 
The soundtrack for each Basic Skill lesson was provided by a cassette 
tape synchronized with the accompanying filmstrip. A complete or near- 
complete set of Basic Skill lessons was available in all schools partic- 
ipating in the Junior High m throughout 1974-75. In the Junior 
High Program, Basic Ski] less. , jre supposed to be the primary cur- 
riculum for Objective I si.. k^see Project application, p. 16). 

The Basic Skill curriculum was not a part of the Intermediate Program 
as originally proposed; however, the Project proposal (p. 54) said that 
"consideration will be given to acquisition of the Minneapolis Basic Skill 
Program.'* By January, 1975, the decision had been made to purchase ' these 
materials and a set of Basic Skill lessons was available in half the 
Intermediate Program schools, 

2. T he Dorsett Readin g Program was the curriculum originally designed 
for use with the Dorsett machines. Dorsett lessons were provided to every 
ESAA school with Dorsett machines. All Intermediate Program schools, except 
those with tutors, had Dorsett lessons for the entire 1974-75 school year. 
The Dorsett Program, more than the Basic Skill Program, emphasized story 
reading in the development of reading skills. Each Dorsett story is graded 
accordintj to the comprehension level it requires. A number of different 
skills may be combined in the same high-interest Dorsett story. In the 
Basic Skill Program, however, each lesson stresses one specific skill. 

Cost of instructional materials and equipment . The Basic Skill Centers 
Reading Program and the commercial Dorsett Reading Program, along with the 
Dorsett A-V machines themselves , accounted for nearly all of the ESAA funds 
budgeted for instructional materials and equipment. In the Intermediate 
Program $40,661 was budgeted for instructional materials, $19,250 for 
Dorsett machines, and $4,500 for maintenance of these machines. 

The materials and equipment budget for the Junior High Program was 
prepared separately for the public junior highs and the nonpublic schools. 
For public junior highs, $25,000 was budgeted for materials, $1,800 for 
purchase of six Dorsett machines, and $1,000 for machine maintenance. For 
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nonpublic junior highs, $3,035 was allotted for materials and $1,480 
for purchase of 5 Dorsett machines. The Junior High Program, in its 
second year of operation, already had most of the needed materials and 
machines. 

Small portions of the materials budgets of each Program were used 
to buy materials other than Basic Skill Centers and commercial Dorsett 
materials. These supplementary materials included commercially prepared 
high interest books with accompanying cassette narrations. Some com- 
mercial Dorsett vocabulary lessons were also purchased for ESAA junior 
highs and nonpublic schools. 

Frequency of Us e ^ R eading Materials . ' 

Tables 1 ano ...jw liiu frequency of use of different reading materials 
by ESAA students in the Intermediate and Junior High reading programs. 
These tables indicate that the mainstays of each program were the machine- 
usable lessons just discussed. The Junior High Program, more than the 
Intermediate Program, struck an even balance between the use of Dorsett 
lessons and the use of Basic Skill lessons. Table 2 shows that in the 
Junior High Program the average^'per pupil use for Dorsett lessons 
and for Basic Skill lessons was about *'One or two days'' out of every five 
days attended. 

Acco:^ding to Table 1, however, the Intermediate Pro, relied very -^^^ 
heavily on Dorset* lessons; about 3-5 days a week per pup on the average. 

Only 9 of the 18 Intermediate Program schools (1,3, 9, 15, 17, 19, 
25, and 26) had h^lc Skill lessons. Each of these 9 schor ^ :received — in 

January, 1975--a complete set of all available Minneapolis asic Skill 
Centers Reading Program lessons. Each lesson set cost $1700. 

Hard to understand, nevertheless, is the low frequency of use of Basic 
Skill lessons even in those Intermediate Program schools housing these 
lessons. Of the 9 schools with Basic Skill lessons, 6 reported an average 
frequency of usage af only "once or tirice'' per student; 2 schools reported 
an average usage per student of ''neve-r this year so far.*' Among the 9 
Intermediate n^rog^ram: schools, only 1 reported an average frequency of use 
as high as ^'1 es? -l:h:: n one day opt of five, but during at least several ESAA 
class sessions." 
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Other reading materials > As mentioned previously, most students in the 
ESAA reading programs were easily able to complete two (but not three) machine 
programmed lessons (either Basic Skill or Dorsett) within a 35-45 minute class 
period. The ESAA reading teachers therefore had the practical problem of find 
ing supplementary activities for the remaining class time. Tables 1 and 2 
show that all but two of the Intermediate Program teachers and all but 
one of the Junior High Program teachers used materials to supplement Basic 
Skill and Dorsett lessons. The average number of different supplementary 
materials used per student was 2.4 for the intermediate ESAA students; 3.2 
for the junior high students; and 3.6 for the junior high nonpublic 
students. 

As shown in. Tables 1 and 2, these supplementary materials included: 

1. Teacher-made materials to teach basic reading skills . After 
Dorsett and Basic Skill lessons, these teacher made materials were most 
frequently used. In both programs the average per pupil frequency of use 
was about *'Less than one day om: of five., but during at least several ESAA 
cIa5S^5essions.** These materials included worksheets, games, puzzles, 
flashcards, and other exercises to teach vocabulary, spelling, phonics, 
"comprehending the main idea in a story,'' writing, and other reading- 
related skills. In two cases, teachers indicated they had adapted a TV 
game show to teach a reading-related topix. Other teachers adapted 
materials from commercial sources: or borrowed materials created by other 
teachers. 

2. Commercially prepared mairerials (other than Dorsett or Basic Skill 
lessons) designed to teach basic reading skills . More than half of t:he 
teachers in each program indicated they had used parts of reading series 
and other commercially prepared materials. Tables 1 and 2, however, show 
the average per pupil frequency of use to be quite low for these commercial 
materials, especially in public junior highs. Only in the three Intermediate 
Program schools with tutors (and no Dorsett or Basic Skill lessons) and in 
one Junior Hi^h Program no npublic school did students use cornmercial 
materials at least once a week, on the average, 

3. Other materials infrequently used by ESAA student.s included 

high interest nooks with accompanying cassette narrations (lESAA funds were 
used to purchase some of these book-cassette series) ; 1 ibrary books 
and paperbacl3 ; popular magazine.s and read.ing-cricnted .scholastic 
magazines; ccirnnercial word games; and newspapers. In only one Intermediate 
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Program school, three Junior High Program public schools, and two Junior 
High Program nonpublic schools were any of these materials used once a 
week or more. 

Multi-ethnic materials . Since the total ESAA Project was designed 
to help solve any problems associated with Minneapolis' school-desegregation 
ESAA teachers were asked whether they had used ''multi-ethnic" reading 
materials, **designed to promote understanding of, arid respect for, different 
racial and ethnic groups" (See Appendix C questionnaire, p. 48). Only one- 
third of the 18 Intermediate Program teachers said they used multi-othnic 
materials, whereas about two- thirds of the 13 Junior lUy^'n luogram ccu^iiers 
re. ponding to their questionnaire said they used such materials. Several 
reachers indicated they had found entire series or workbooks devoted to 
multi-ethnic themes (e.g.. Black history). Several other teachers 
apparently searched Icmg and hard for mml ti-ethnic materials, with varying 
success. One teacher found a book on 'tetive Americans at the Minnesota 
Historical Society;: another found the school library a useful resource. 
Only one teacher sxi^gested that the Basic Skill and Dorsett lessons ful- 
filled the definiti'on of "multi-ethnic materials/- 

ESAA Reading Students: Their Selection and Character istics 

Projected numbers of service-eligible students for the ESAA Reading 
programs . The proposal for the 1974-75 Minneapolis ESAA Project estimated 
the numbers of students that could be served by each reading program, given 
specified criteria for instruction-eligibility (see pp. 5-7 of this report 

The application for funds estimated that 17 of the 18 Intermediate 
Program schools could, as of fall, 1974, each expect to find 137 students 
eligible for ESM instruction. The basis for these projections was an 
unspecified ''needs assessment'^ (Project application, p. 53). 

The Project application (p. 13) used results of 1972 and 1973 citywide 
testing to estimate that each Junior High Program public school could 
expect as of fall, 1974, at least 263 students eligible for ESAA instruction 
according to the above criteria. Each Junior High Program nonpublic school 
coixld expect such students. 

Selection of students for the Intermediate Program . A combination of 
test scores, recommendations of classroom teachers, and other factors 
spezdfic to each school (e.g., Reading Coordinator judgments, availability 
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of space in the ESAA Reading Center, etc.) were used in an attempt to 
select students according to the ESAA Project proposal's criteria. 

As noted on p. 5 of this report, tho criteria for Intermediate 
Program eligibility unexpectedly generated too few students at each 
school. In addition. Intermediate Program teachers at the start 1974-75 
had only Dorsett reading lessons, which presuppose some decodir. ^ vills. 
ITiese teachers could not therefore 1. aJle Uyr p^^orest reading stuuuii ls 
in t-hcir school; i.e., those intermediate grade students two or more 
years below grade in comprehension. In early fail, 1974, therefore, 
the Intermediate Program began to seek students with sosne decoding slcills 
who were one or more years below grade level in compreh^sion. 

Actual number of students served by the ESAA reading programs . 
Table 3 shows that 1900 students were served by the coicbined ESAA reading 
T^JTOgrams. A total of 989 were served by the 18 Intermediate Program 
schools, for an average of 55 students per school. The 8 public Junior 
High Program schools served 758 students, or 95 per school. The 5 
Tiionpublic Junior High Program schools instructed 153 students, or 31 
per school. 

Per pupil cost of the programs . Based on the amounts budgeted for 
each program (see p. 2 of this report) , the Intermediate Program cost $331 
for each enrolled student; the Junior High Program in public schools, $309 
per student; and the Junior High Program in nonpublic schools, $223 per 
student. 

Pretest reading comprehension . Table 3 indicates that the mean pre- 
test grade score was 5.2 for Intermediate Program students; 4.1 for Junior 
High Program students in public schools; and 4,9 for Junior High Program 
students in nonpublic schools. The standard deviations of the pretest 
means for each school were generally near 1: in general, about two 
thirds of the ESAA students at each school had pretest scores between 1 
grade-score year above, and 1 grade-score yea)- below, their school's 
ESAA mean, ' 

Guidelines for the Junior High Program specified that no student would 
have a pretest grade score greater than 6.0, In fact. 48 (6%) cf the Junior 
High Program public school students and 37 (24-o) cf the Junior ILgh Program 
nonpublic school students had pretest grade scores greater than 6.0. These 
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Tahiti 3 

Selected Characteristics of ESAA Reading Students 
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Note. — Statistics for each variable in this table are based' on students with complete data for that variable. 
For each variable the total number of students is not footnoted unless this N is at least 5% less than the 
total number of ESAA students. 

^19% of the Intermediate Program students were enrolled in general academic programs classified as "ungraded 
upper elementary." The schools and their percents of ESAA students ungraded were School 1 — 56%, School 4 — 100%, 
School 15—88%, and School 25—98%. 

^16% of the Junior High Program nonpublic students were upper elementary students. The schools and their 
percents of ESAA students in elementary grades were School I — 11%, School 1 — 36%, and School M — 41%. 

^An additional .9% of all ESAA stndents In borh pr -^rams were Spnnish-.sumamed , .4% were Asian American, 
and .7% were designated "Ottier" race or ethnic group.. 

Months in the program is defined as the number of enrolled days (including absences) between the pretest 
and the posttest, divided by 17.9, the average number of days in a school month. 

^This statistic is based on the number of students riding a bus as part of the Minneapolis Public Schools' 
desegregation plan. Nonpublic schools did not participate in this plan. 
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"N=77 
^N=84 
^=10 
^N=50 
-^N=28 



'■N=683 
^=103 
"n=^76 
^N=58 
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^N=141 
^N«30 
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stuiJents were served by ESAA reading programs even though they were in- 
eligible . 

For the Intermediate Program, it is more difficult to determine the 
number of ineligible students that were in fact served because 19% of 
the Intermediate Program students were enrolled in schools or general 
academic programs classified as "ungraded upper elementary." For such 
students the discrepancy between grade placement and grade score could 
not of course be computed. Among Intermediate Program students who could 
be classified as 4th, 5th, or 6th graders, however, 38 (4-6) should not 
have been enrolled if the "one-year-below-grade-level" revised eligibility 
criterion had been strictly followed. 

Months enrolled and attendance in the ESAA reading programs , "Months 
enrolled" was defined as the number of enrolled days (including absences) 
between the pretest and the posttest, divided by 17.9, the average number 
of days in a school month, (During 1974-75 there were 179 school days.) 
For the Intermediate Reading Program, the average number of months enrolled 
per student was 4.4 months; for the Junior High Program in public schools, 
5.4 months; and for the Junior High Program in nonpublic schools, 6.6 
months. For students who remained in their ESAA reading program until the 
end of the school year, the number of months enrolled is underestimated 
because the timetable for the conduct of this evaluation required completion 
of all testing by mid-May. 

In thre Intermediate Program, 71% of the students took their ESAA pre- 
test during September, October, or November; 70% took their posttest in 
April or May, While enrolled, the attendance of these students in the ESAA 
class was 82°a (standard deviation = 15%). 

In the Junior High Program, 72% of the public school students took the 
pretest from September through October; 68% took the posttest in April or 
May. Among the nonpublic students, 81% took their pretest in September or 
October, and 84% took their posttest in May. Th.e^ESAA- class attendance of 
Junior High Program public school students was 85% (s.d. = 15%); the 
attendance of nonpublic students was 90% (s.d. = 11%). 

Other notable student characteristics . In both the Intermediate 
Program and the public school Junior High Program (a) boys somewhat out- 
numbered girl£,, (b) the proportion of Black students was about one- third, 
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(c) the proportion of Native American students was 10%, and (d) a majority 

of students rode the bus to school as part of the Minneapolis Schools' 

desegregation plan. In the nonpublic schools (a) girls outnumbered boys, 

(b) about one -fourth of the ESAA students were Black, and (c) about 15% 

were Native American. * 

The Staffing and Daily Operation of the Reading Programs 

Staffing . The Intermediate Program's instructional staff consisted 
of 18 teachers and 16 teacher aides. The combined public and nonpublic 
Junior High Program had 15 teachers and 17 teacher aides. 

Not all of these personnel worked full-time, however. For most o£ the 
school year, the Intermediate Program had funding for 15.5 full-time- 
equivalent teaching positions, 3 reading tutors at 3 hours per day, and 
16 full'time-equivalent teacher aide positions. The three part-time 
tutors in the Intermediate Program (in Schools 4, 20, and 22) worked with- 
out benefit of the teaching machines and many of the materials provided to 
the other ESAA intermediate reading centers. 

During the entire 1974-75 school year, the Junior High Program had 
funding for 9 full-time-equivalent teaching positions and 11 full-time- 
equivalent teacher aide positions. All 7 nonpublic Junior High Program 
teachers worked. half-time or less with ESAA funding. 

Among the 33 aides in both ESAA reading programs combined, about one- 
fifth worked half-time (4 hours per day) or less with ESAA funding. About 
four- fifths worked as ESAA aides for a full school day (between 6 and 7 
hours) . 

Staff characteristics . The staff in both reading programs were predom- 
inantly White and female. Among those 31 of 33 teachers answering the ESAA 
Reading Teacher Questionnaire (Appendix C) , 26 (84%) were White and 24 (77%) 
were female. Among, those 32 of 33 aides answering the ESAA Reading Aide 
Questionnaire, 26 (81%) were White and 30 (94%) were female. The only 
minority group apparently represented among the teachers or aides was Black 
American, although two aides did not identify their racial/ethnic group. 

Among teachers, almost one-third (29%) had a master's degree, about half (46%) 
of the public school teachers had tenure in the Minneapolis Public Schools, 
and about- one -third (35%) had Minnesota teacher certification in reading. 
The Junior High Program public-school teachers appeared best qualified to 
teach reading: 7 of 8 had. reading certification. Only 3 of 15 Intermediate 
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Program teachers and one of four respondents among nonpublic Junior High 
Program teachers held reading certification. 

About half of the teachers in each program had taken four or more 
courses in reading. The mean number of reading courses was 4.1 for the 
entire responding group; the standard deviation 2.9. Four teachers in 
the Intermediate Program had taken no reading courses as of May, 1975, 
the date of the teacher questionnaire. 

Daily operation. In each ESAA school with Dorsett machines, one 
classroom was designated as an ESAA reading center or reading lab. While 
enrolled in an ESAA program, nearly all intermediate and junior high 
^ students were scheduled into the reading center at least four times a 
week. Nearly every full-time public school teacher had a full class 
schedule of ESAA classes. One teacher funded for full-time ESAA work, 
however, inexplicably reported working half-time with ESAA students and 
half-time with non-ESAA students. 

For ^any ESAA students, the ESAA reading class substituted for social 
studies, English, or at the intermediate level, for reading instruction 
by the regular classroom teacher. 

While visiting ESAA classes, the evaluator found students generally 
active, attentive, and productive. As they entered a machine-equipped 
reading lab, most students began their Dorsett or Basic Skill lessons with 
a minimum of verbal instruction by staff. On completion of a lesson, 
students completed a short written exercise which they took to an aide or 
teacher for scoring. Students then either returned to the machine for 
another audiovisual lesson or began one of the supplementary reading activ- 
ities described earlier in this report. In about half of the schools, 
teachers also involved students in some type of writing exercise during at 
least several class sessions. 

Discipli ne and the E SAA programs . The generally quiet, orderly 
atmosphere of the ESAA class suggested to the evaluator that a reading 
program with audiovisual teaching machines might be particularly well suited 

„P.^95_ who are also disruptive and hard to manage in a regular class- 

room. About half of the ESAA teachers responding to the teacher questionnaire 
also felt that the ESAA program was i)articulariy appropriate for such students 
however, one-third Indicated they didn't know whether the FiSAA program was 
particularly appropriate for disruptive students (see p. 45). 
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When asked,. "What proportion of your total ESAA student group came 
to the ESAA Reading Center mainly because they were disruptive and hard 
to manag e in other classes , and not because they had severe reading problems?'' 
9 of every 10 teachers answered, "Very few..." or "None..." (see p. 45). 
Only one teacher felt that "About half" of the ESAA students were mainly 
discipline problems and not poor readers. In short, the ESAA programs do not 
appear to have been "dumping grounds" for teachers' disciplinary cases. 

Working relationships between ESAA reading teachers and aides . These 
relationships were reported "Very good" or "Excellent" by nearly all of 
the aides and teachers. Three-fourths of both groups, teachers and aides 
used. "Excellent" to describe this working relationship (see pp. 45 and 49). 

The role of the teacher aides . When asked to list their three most 
important job activities (see p. 50), ESAA aides most frequently mentioned 
the following activities: Encouraging or praising the student; Working 
directly with students and helping them with their work; Maintaining up-to- 
date student records; and Organizing and storing the reading materials, 
and preparing them for use. Only a few aides and two teachers indicated 
any need for change in the aides' role. 

Although three-fourths of the aides paid they enjoyed their work 
"Very greatly" (see p. 49), about one-fifth of the aides indicated some 
dissatisfaction with the conditions of their employment; e.g., pay scales. 
Civil Service seniority regulations, job insecurity, etc. (see p. 49). 

A time analysis of the Junior High Program reading aide's role was 
included in the previous year's evaluation report (see Higgins, 1974c, 
pp. 15-16). 

The Dorsett machine: again, some mechanical problems . This year, as 
last year (see Higgins, 1974c, p. 17), some teachers and aides had complaints 
about the mechanical operation of the Dorsett machines. About half of this 
year's teachers had dissatisfactions with the machines or their servicing 
(see p. 46). Five teachers said the machines were* too easily broken (e.g., 
"wiring too easily accessible"); 5 teachers cited mechanical difficulties 
(e.g., "trouble with film advance," "errors in sequencing"). Six teachers 
mentioned inconvenient features that suggest needed improvements (e.g., 
"needle difficult to maneuver," "earphones don't fit ... ," "machine should 
allow student to reverse filmstrip. .. without losing place," "light bulbs 
burn out too quickly," "replace records with cassette tapes"). 
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The evaluator has already advpcated (1974c, p. 261 that a time-delay 
feature be added to the Dorsett machine, so that a student answering 
incorrectly receives not only a 1-second "error tone/' but also a 10 or 
. 15-second ''time-out'' period when further responding cannot advance the 
filmstrip. The evaluator's observations and teacher comments suggest 
that the error tone is not aversive for many students. Such students 
sometimes carelessly press the machine's buttons until the correct answer 
is located, and so progress through the lesson without reading it. A 
time delay for incorrect answering would eliminate reinforcement for 
random button pressing and therefore make such careless responding less 
frequent. 

This year teachers were asked if they would favor a time delay for 
incorrect responding (see p. 46). Over three-fourths of the teachers in- 
dicated they would favor such a change in the Dorsett machine. 

Training in the use of Dorsett machines > audiovisual reading lesson s, 
and other reading materials . Teachers and aides generally found their 
pre-service and inservice training sessions to be at least moderately 
valuable (see pp. 48, 49)- Both teachers and aides made a number of 
suggestions for improved training including: (a) More sharing of ideas 
and materials between teachers (suggestion made by 6 teachers); (b) More 
sessions on adapting Dorsett and Basic Skill lessons to individual needs 
(7 teachers and 3 aides); and (c) More formal academic training in such 
topics as child psychology and remedial reading (6 aides and several 
teachers) . 

Expressed desire for use of a diagnostic reading test instead o f a 
survey tesc . Teachers were asked tp indicate the testing procedure they 
would most favor if the ESAA reading programs were continued beyond 1974-75 
Three-fifths of the teachers indicated they would prefer a diagnostic 
reading test that attempted to measure different types of reading compre- 
hension and other reading subskills (p. 46). Only 3 teachers said they 
would prefer retaining the Gates-MacGinit ie survey-type tests used during 
1974-75. Four-fifths of the 31 responding teachers said they would be 
willing to spend 2 or 3 hours scoring and profiling a set of 30 tests (this 
time estimate based on the manual for one popular standardized diagnostic 
test). In response to another quest ion--one requesting general suggestions 
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for program improvement — 10 of 19 teachers responding suggested the use 
of diagnostic tests or the testing of specific reading subskills (see 
Question 60, p. 48). 

Other remedial reading programs in ESAA schools . Most of the public 
and nonpublic ESAA schools had other programs besides ESAA to teach basic 
reading skills (see Question 31, p. 45). These other reading programs 
included SLRP (mentioned by teachers in 18 schools). Title I (mentioned 
for 12 schools), Basic Skill Centers (students were bussed to these 
centers from at least 6 ESAA schools), other special education progrsims, 
and tutors. 

The evaluator does not know the degree to which the ESAA reading 
program and these other reading programs were coordinated. In response 
to the request for general suggestions (see Question 60, p. 48), two 
teachers expressed a desire for greater coordination of their school's 
reading programs. 
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THE EFFECTIVENESS OF THE ESAA READING 
PROGRAMS IN MEETING THEIR OBJECTIVES 



Did students enrolled in ESAA reading programs during 1974-75 make 
those gains in reading comprehension stated in the objectives for each 
program (see pp. 5-7)? The objectives for the Intermediate Program 
stated in part that the median gain would be 1.0 grade-equivalent month 
for each month enrolled. The objectives for the Junior High Program 
stated in part that (a) Objective I students, who entered the Program 
with comprehension levels of 3.9 or-below, would make a median gain of 
1.5 grade -equivalent months for each month enrolled, and (b) Objective II 
students, with pretest of 4.0-6.0, would achieve a median gain of 2.5 
grade-score months for each month enrolled. 

Table 4 shows the reading gain rates made by students in the Inter- 
mediate Program. Table 5 shows reading gains made by Objective I students 
in the Junior High Program. Table 6 shows the reading gains achieved by 
Objective II students in the Junior High Program. These tables show 
reading gains both for individual schools (if they had 19-20 ESAA students) 
and for total groups of students. For the Junior High Piogram, separate 
totals are presented for public vs. nonpublic schools. 

Tables 4-6 also show (a) the degree to which students were inappro- 
priately tested, (b) the extent to which students ineligible for ESAA 
instruction were actually served, and (c) the proportion of students for 
whom data needed to compute reading gain were missing.^ 
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to H f the guidelines of the Intermediate Program, students were 
to be tested using either a Gates-MacGinitie CI or C2 Comprehension Test. 
Instruction-eligible students were those at least one year below grade place- 
ment m comprehension. Table 4 shows that of the 989 iotal students in the 
Jn^r V "^^^ inappropriately tested with Gates D 

Im^r/.r rIo^"" ' "'"'^""^^ -"i^^i^g gain-rate data. 

Among the 889 appropriately tested students, only 7 (1%) were clearly in- 
eligible for service Another 201 (23%) were enrolled in academic programs 
classified as "ungraded upper elementary," and their eligibility for instruc- 
tion could not therefore be determined. mstruc 

The guidelines of the Junior High Program specified that Objective I 
students were to be tested using a C2; Objective II students were to be 
tested using a 02 or D2M. No student in the Junior High Program should 
have had a pretest comprehension grade score greater than 6 0 

c '^t^i^rJof'^ ^ ^"^^ "ig*^ Program' had 9il ' students. Of 

these 342 (38^) were Objective I students, 394 (43%) were Objective II 
students, 79 (9%) had pretest scores above 6.0 and were therefore served 
though ineligible, and 96 (11%) were missing some data item needed to 

t\arC2"c2'or"Df'Sr ^^^^^ pretest-posttest combinations 

aJd 20 Ob?i^iJ^ ?^ .I'f'. ^^'^'"^ ^^'"^ °^ 22 Objective I students 
ana Ubjective II students were not computed. 
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Attainment p--' '-rjectives b;. the In.. -diare ' .^jv-g Program 

The Inte i-mediare Reading ProgKi r ot oniv br.t surpassed its goals 

for coiim'-hsTT::-ion ga-in among disable- aader? fele 4 shows that Inf ;r- 
mediate Pr: ^riim stucL£)nts who tooL a js C2 fcrr ^-olh pretest and p.o'-^ -.test 
made a mecian of sli-htly over 3 mon-h; compr?r ,cn>--i dh gain for each 
month enrolled in ths Program. Students who to. • C2 for their T.-rer.. 
and CI for tr.- r pc test had a :Bedi£;- gain -3=.: of 2.3. 

These 8S^ Epr-r-rnriately rested students JS in the CZ-C2 grcrav ' 
171 in the C2 - Zl g— 3ap) together consrrLtuted 90% m all stuidentjj sar.vsd 
by the Interme±iate Program. The 681 students clearly eligible for 
service const^iited 69% of the tolal fnt -rmecrate group. Although clearly 
instruction-e_igibiie C'Elig.") sfadeirts were only three-fourths as 
numerous as the total greup of "All" appropriately tested students, witr^in 
a given test group (C2-C2 vs. C2-C1) the median and mean gain rates were 
nearly identical for the "Elig." and "All" categories. 

The objectives of the Intermediate Program also specified catefor of 
gain that would be attained by different proportions of students: ZSl wers 
supposed to f^ll into the iiighest category, K4 or more months gain per 
Program month: 25% were to gain at the rate of 1.0-1.3; and the reiEainiii^ 50% 
werr? expected -o gain less than 1.0 month in neading skill per Prograi?. month. 

As shown in Table 4, regardless of the tetsting procedure used, afaout 
thrais times as many of the Intermediate students as expected fell intc the 
higr-est category of gain rate. Again, the use of the same form for pre- 
test and posttest was associated with higher proportions of students aieet- 
ing the objectives than the use of different, ecaivalent forms. 

This general patteTn of reading gain result-- for the Intermediate Program 
obtained for both (a) all students, regardless o.r" pretest score; and fb) 
instruction-eligdbltf stiudents only, whose pretest grade scores were at least 
one year below tueix grade placerant at pretest time. 

Also, this pattern of re.suliL., held not only for the total Intermediate 
group, but aisc for individuaJ school . Each Irtixermediate Prograim scnool had 
a median gain r-te at least twice- the 1.0 that wes expected. Each scnool had 
two-thirds ox mo-e o~ irs students in rxe highest gain rate cate::5or: . 1 .4 cr 
more months gain pszr Program month. Pund ^jhen separate groups ot" 20 or .-more 
students withic a schtiol had a C2-C2 tasst sequemce vs. a C2-C1 test sequence, 
the C2-C2 group; fared generally better than the C2-C1 group on indict- of 
reading gain. 

27 
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3i.:Tference aifaong schools in avfexat^^ y^ain rates ^ The ev;aiuative 
companstJE: of air.teT3g« ^in r^tes among Tr":?=rmed3 at'?^ Program schools rousr 
be appr:' iched v xh ;Ru:ion. 

Fir<^t, siri; 2 ail schools rset their /f.^^-ectives for reading gain, 
schools :ha:t or.-up;'' r ' e lower ranks in ^ ^tribut^on of meaj\ gain rates 
should r nec!r55ari: ^ evaluated neg&r,. sly. 

Sec:nnc, the gsjin "rate statistic seer's^ unstable, even for subgroups 
of students. T^ii r-snjie of mean gain scorr l- is large. If one ignores 
differeaces amaizc ^:■'i^K^:^Qups in tests ^sed^ and in eligibility for in- 
struction, the h±:i^'j-?:r mean gain rate by rsny subgroup in Table 4 was 7.5, 
for the T: appropms:-::^/ tested (C2-C2) s^dents at School 25. The 
lowest rate was . . imade by both the 36 ZZ-Cl students at School 1 and 
the 20 instructicjn -eligible C2-C2 stnidenrrrs ar School 15. 

The i^ubgroup stamdard deviations are also genersLLly large. The 
overall standard devlaticn cnf scores, 4.t includes tJie value zero^ even 
though only 6% of students h^ad gain rates of zero or less. 

The distribunrion c£ incii vidua! student gain rates is positively 
skewed by small irumfaer^:: of very high scorers. For nearly every subgroup, 
the mean gain ra'rss exceeds the median, ua-aally by at least oiae-halJt a 
month. The posirive r,,;ew of gain rates ?;eems attributable to the 14% of 
Intermediate '^.tiicients .with inear-pferfect scores of 43 or more correct answers 
out of a possi 48 cm the ^.Sates C postrtest. These raw scores corresponded 
to grade scOTei: of 6.0-7.0. 

Despite ri^ hazards in doi-:; schools with relatively h7>/\ and low 
mean gain xa-^^^i will be sin^cl^d our later discussion. The ziniinx rate 
of compreihensi^OT ga^n over aill tte<t amc xnistruction-eligibility groups in 
the Inteni^ixit^; ^Togrann was 3.7, Fair: schools — 8, 25, 26, and 27 — had mean 
gain raeltJs at vL^tSvr ^e grade-tscoirss mnnth: higher than the overall mean. 
These nuiic/ol^- MtrlT. ne callfsd '"h±gher-garnr schcools.*^ Three other schools--3, 
9, an..:: IS^-hiiki ::2iHair. gaiji rgnrefs azL ^-xis^t aire grade -score month lower than 
the overall mss: These schnoLs be :zalled *4ower-gain schools.** 

Attaxjiment oif (IIb'jejx::ivei? by Tthe J^.inicrr H'iigrh Reading Program 

Two set5 :cf o^mectives iwere establisreed for the Junior High Reading 
Program. ObTecrive Set I apnlied to tho^r 342 students entering the Program 
with pretest scorss of 5.9 onr less. Objajrtive Set II applied to the 394 
students with prei^est scores of 4.0-6.0. 



Objective Set : wai met by bor-i th .^^ pub lie- school 

Objective I students anc the total ^to^j )f \(/^-puhii c ^chool students. 
Objective Set II was met by the tot,ai ^ppr^^T^rUte publdc school 

students but not by the Objective 11 s'*:::^^^^^^ nonpublic schorois . Both 
sets of objectives were met in a major^-/ in^^i vid-Jn Junnor High 

Program schools for which reliable iniXc. % ^&s.ding g^itr rate cou^j be 
determined. 

Objective Set I. Table 5 indie t^^ \f ^^blii;->5Chcycx Object, 3 
studeTits made a median rsin of 1.7 i^ -'^^ / ;^'lpreHe>3Sijyii gain per 
month in the Prorram; therefore, tht; oP'uA^j^^ .^Mian 3pp:fo.xamaled cne 
median of 1.5 sp©::ified in the obje-'ive^ . No^Public-sciiQQl Objecr:.ve I 
students gained a. a rats of 4.1, a st^t>\^C ^iiuinished .in import-imce 
beca-ise it was bLsed on only 10 stucleij-i^ii.^ 

Objective Ser I also specified ho{?e^^/ j^/Vrtions Of students m 
various categories of gsim rate: 30?« ^^xe fe/'^e 2,5- o^- nnore months gain 
per Program month:; 15% were to gain -^^ ic^^ o/ 1-7-^.4 5 3o% were to gain 
at rates of 1,0-1.6; and the remaining 25^ to i^l beiow £ 1,0 rate. 

In fact., the rate distribution obt-dnec^- ^0 ^pprocrri^tely rested 
public-school students very closel; 3.^^\o\^\^^ the ciastribution specif, ed 
in the objectives. The discrepancies i/^^ hop&d-fOT proportions 

•:ind the obtained pronortior-. range'.^i -xo\ -0:; \d> <12. In no^xipubUc schools, 
3 of the 10 appropr:.3te:y teisted Ol 3'-<i.v^ r St^^^^ents wfjfe f.n th ; niptrest 
:;ain-rate category. 

Objective Set IT. Table 6 she : ■ ^lA^^^^ school "bjecti.-e I: 

students nade a mediiffi gain of ""v; >fcP> "^Ming the Haped-f-.ar rn^ciiin 

of 2.5. The obtained ne.>dian or :.s tror tPi\^^k~? scJ^ooi Objectii - ir 
srudents failed to mset tvzzs a-bjscxi- o \^ ^°kt\^5 gaiin per PrccT^n.-, morfh. 

Objective Set r: al.sc spi ;:fisc thAi c?'^ the Objective II s'udenrs 
-should make 4.0 cr mere mcnths .^iain per %nth; 15"^ shoulc make 2.7- 



Table S presents resLults separately \yy ^^i- tMo^e schools v-.th 19 
or more Objective I studenns; Table 6, c^^l-y ^^x" ^cho^^l? wiih 20 or ma--c 
Objective II studenrs'. Re:3ults based oD A^,^t ^^tudeUt^ ve^e judges- tcao 
unstable for separate tabulation. In z<A&^\(p\ ^eq^^ucy distribu.- loms 
(and for appropriate! v tr -ed s iudents, 'I'^X^f '^^^^riV^iv^ s^azistz^rs) 
are presented only f ■:. r thu^so CZ and Q2^\ t^^^ sut'^arrou.ps nmnbermg 
8 or more. The resu-t- for -ot.-. l group^ f V ^^Ub^^c aT^i 7ionpubiic_^-tade ,ts 
in Tables 5 and 6 nasseci an all such s^\^^t,^ VitM complete trst ar 1 
enrollment data, ev-sn £-:ude-nts whose Szn^O\ k^^e omitted fj-om t-Tula- 
presentation. 

'h 



3.9 months gain per Program month; 50% should gain at ratss of 1. 0-^2. 6; 
and the remainder would therefore gain at less than a l.C rate. For 
public-school Objective II students, the obtained rat:2 distribution 
closely approximated the distribui:::LOn specified iit the a!?:?ectives. 
Discrepancies between obtained and hoped-for proporticns in each gain 
rate category ranged from .05 to .12. Ir nonpublic scnDa2:is, however, 
nearly half of the Objective II students fell into the I, .0-2. 6 gain 
rate category, a fact accoimting for the low median rar:e smong this grcup. 

Differences amiong schools in a tt ailment of reading : :::7bjecrives. Urn- 
like the Intermediate Program schools, nt^ every Junior .-High program sirhool 
met its objectives for reading gain. Th^^ ^valuator sougtnr; sonne reasonable 
criteria to differentiate those junior irigfri schools that ^lad niost success- 
fully met their ESAA reading objectives^ rrrom xhose thar: kad least success- 
fully met their objectives. In the evalcistor * s judgmenrt, clear evidence 
that a particular school had or had nor me^ its reading cirT^jecrives c ^uLc 
exist only when the school ""s ESAA ins-rr ac^niun r^srveid a s . :.^aii3le group of 
students and when those students had also been appropriaite^ly iiestsd witin a 
C2 or D2. The evaluator therefore deci'^jed Xo nvsike ^mala'ti^irt.ire judgnseirt? 
regarding snccess only among junior high>^ (a'^ having Obje^crtive I :rr Qhne^txw 
II groups each numbering 19-2C' and (b) hiav i^ig les:5t " v^jjectivre 1 3rr;::cent-5 
or 8 Objective II students wht were 3ppr^a:'i:aa:edy "-:^5ted. Usrng th^s:ir 
criteria, only 3 of 8 public junior higns-- Scfmols A, f ., ;md F--.cre2ril7 i^et 
Objective I. Four public jujiror highs — E. F, amd H— JiJ.^arliy nnet ''-ki^'^ztive 

Only in School F did appropriately -nre^^usd ObT::ntiAT^a? Z and OibjecTiv.r II 
students meet their respective sets of ab^^^r::rtives . Alsjo. fonly for Sclio^'.: G 
was there clear evidence that Objective ^ amd Objeciriwa H students hac 
both flailed to meet their objectives. 

Factors Possibly Accounting for Differsncefi: Among Sz.uc ^eiits and Sch ocis 
in Reading Gain Rates 

In this section a number of faotrrr V\ mught plc;ii::o;ir[ y accourr ' f- — 

differences in reading gain will be crviisridsred. To arx-r:^: ipate the cJ 

cussion, only two factors — variations ir testing: prars^ ajzce and Icngt 

of ttime in the program- -were importanitly rel.ate.ci tc Tieading gain 

rates , 

The following additional factors were cGrnsixiersd sersirately fo-r 
students in the Intermediate and pubLic school JuunixcT HItJi programs and 
found to have no important relationship with gain rc:re: ' ne number rf 



supplementary materials used, the frequency of use of teacher -made mt^^^^i^^^ 
pretest grade score, percent attendance while in the Program, student' ^ A/^i 
placement leveiL, and whether or not the student was bussed to school. 

An "imporrrant relationship with gain rate" was defined as a Pears^t^p 
correlation between gain rate and a given variable accounting for at 
least 6% of the variance (r=,25 or more, ignoring sign). 

Even the frequency of use of Dorsett and Basic Skills lessons was 
not consistently related to gain rate. For Intermediate Program studeJi^t^, 
gain rate was uncorrelated with the number of times per week students 
used Dorsett or Basic Skills lessons. ' In public Junior High program 
schools, however, students with the highest gain rates tended to use 
Dorsett materials mere frequently and Basic Skill lessons less frequenii^ijr 
Cr's^.37 and -.46 between gain rate and the frequency of use of Dorset^ 
and Basic Sf ills lessons, respectively). These correlations, however, 
probably do not indicate that use of Dorsett lessons promoted reading 
gain more trian ujse of Basic Skill lessons. Instead, these correla- 
tions probably indicate only that students having the greatest difficult/ 
learning to reaa: (those with the lowest rates) tended to be assigned 
Basic Skill lesF;ons, whereas students having less difficulty (high rat^ 
students} tendei to be as Aligned Dorsett lessons. 

Agaii the c valuator can point to only two Program factors--testin^ 
procedure and duration of ESAA enrol lment--that have important, indepet#^ 
dent relatianshi ns with reading gain rates. 

The effect o f testin g procedure variations on measures reading ga L^. 
As noted above, rhe evaluator had in the previous year's evaluation of 
the ESAA "eading Program (Higgins, 1974c) suggested the use of differeivl^^ 
equivalent test forms for pretest and posttest to achieve Rebalanced 
research design (p. 26). The basis for this suggestion was some studei,1^s 
whose reading gains were implausibly large considering their short stay 
in the ESAA reading program (pp. 21-22). The evaluator suggested that 
such results might be attributed to remembering comprehension passages 
from pretest to the identical posttest. 

This suggestion for alternate forms was not followed, and this yea:^ 
as last, ESAA students generally used identical forms for pretest and 
posttest. The only exception was the Intermediate Program, where subgjc^ii,,^ ^ 
of students in Schools 1, 7, 8, 13, 14. 15, 17, 19, and 25 took C2 as a 
pretest and CI as a posttest. Other subgroups in the same schools took 
C2 for botn pretest and posttest. 
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By comparing the gain rates of C2-C1 and C2-C2 subgroups in schools 
where both pre-post test procedures were followed, the evaluator hoped ^ 
determine the effect on these rates of taking alternate vs, identical test 
forms. On the basis of the previous yearns evaluation cited above, the 
evaluation suspected that reading gains of C2-C2 students might be higher 
than those of C2-C1 students. 

The comparison of gain rates of C2'-C1 and 02-02 students is clouded, 
however, because the CI Comprehension Test is apparently slightly more 
difficult than the C2 Comprehension Test. 

; The Technical Manual for the tests (Gates S MacGinitie, 1955b) describes 
C Form 1 and C Form 2 as "paralled forms" (p. 7), For norm samples of over 
1000 5rd graders taking each 48-item form, however, the Form 1 Comprehension 
Test (mean raw scores=23.2) was 1.5 correct answers more difficult than the 
Form 2 Comprehension Test (mean raw score-24*7). While the standard de- 
viations are the same (10,8 for Form 1, 10.7 for Form 2), unless the means 
are also equal, the two tests cannot, strictly speaking, be called parallel 
or equivalent tests. 

The test authors ignored the apparent nonequivalence of the CI and C2 
Comprehension Tests in constructing both (a) the raw to standard score 
conversion table and (b) the raw to grade score conversion table (Gates § 
MacGinitie, 1965a, p- 8). For a given Comprehension raw score, these con- 
version tables assign the same grade score (or the same standard score) 
whether the student took the CI or the C2. 

To attempt a fair comparison between gain rates based on C2-C1 vs. 

C2-C2 testing, the evaluator added 1.5 raw score points (correct answers) 

4 

to each Cl posttest score. The test authors' raw to grade score conversion 
table was then used to compute Comprehension gain, and then gain rate. 
The evaluator' s corrections resulted in from 0 to 6 months being added to 
each Cl grade score; 43 of the 48 corrections were included in the range 
from 0*5 months to 4 months. 

Table 7 compares C2-C2 Comprehension gain rates with C2-C1 gain rates 
based on corrected vs. uncorrected Cl scores* If Cl raw scores are ac^ 
curately corrected by adding 1.5 points acorss-the-board, then Table 7 
suggests that it matters little whether identical or different forms 
are used to measure gain rates. 

The evaluator is indebted to Lary Johnson for pointing out the non-comparability 
of C2-'C2 and C2-C1 gain rates. The evaluator, however, takes responsibility 
for , the method of correction used here. 
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The 400 C2-C2 a gMn r^te of about 4 grade score months per 

month enrolled; A M ^^'CX students, ^ corrected gain rate of 5.6 months 
per month. 

If. howeve^^ t^^^test authors assert. CI and C2 are in fact equivalent 
tests, and if (^^^f>^^^ar^ ij needed to equate CI and C2 scores, then read- 
ing gain rates tdentic^ai pr^- and post-test forms may be importantly 
higher (0.7 mn^^f „A -^'inth. ^cco^ing to Table 7) than reading gain rates 
obtained using c^^f ^f^^"^ ^est ^orjns. 

Based on tf\i^ ^^^j ^^^cale investigation of gain rates based on identical 
vs. alternate px>,.\fx ^^^^ting, the ^Valuator is reluctant to conclude that 
the use of alte^^^,A, h^^^ in p^e^^o^t testing is generally unimportant. 
Given the circuieisf^^^^s ^tmilar W t^ose of the present study, with C level 
testing of stude>,t^/tt^aJi initial average comprehension levll of about 3, 
however, the us0 ^^k^^^^e p^e^po^t forms may be unnecessary. 

A more adecii,^^, of the in>pc;^tance of alternate form testing would, 

of course, have \eC\ ^ncJ c2 forms (as well as Dl and D2 forms 

when appropriates ^txJdent^ tested) for both pretest and posttest. 

assigning one or A ^t}^^> test to students in a balanced design. A 
comparison of Cl (^^ Dl m D2) pretest score-s could also help decide 

whether or not t\ t„P /^hns ar^ equai in difficulty. 

In the pres^ni? j^v/^Ugaticjn, th© ©valuator did not monitor whether 
the ESAA teachers (^t^,^ instructed, randomly assign Cl and C2 posttests 
to their students, ^fx, .C^^ci ar^d C^-C. test groups did, however, seem well 
matched in the St^andard deviations of their pretest scores. 

One implica«;icr)i^f /T^ble 7 is th^t gaio rates, when calculated as 
required "y this e/ V^t^'^rt design, a^e highly sensitive to slight dif- 
ferences in the <Ai(\^\if)f of suppo^ecily equivalent tests (or to slight 
changes in the ^c^^i \f ^ test). 

, In effect, 'V^\j\ ) ^Hg-^ests tli^t an^ factor (including scoring error, 
remembering prev^.^^i^^ a/'Verecl it^%, or true gain in reading competence) 
that adds 1.5 v^j^\,i^^.^^\hC\ r^w pc7St-score will automatically add 0.7 
months to the mos,^ ^V^ir^S jain ^«ate, if these rates are based on C2-C1 
pre-post testing. 

Given the la^^g^ y^^j/^ili ty in reading gain rates shown in Table 7 
(and elsewhere in tHi^ ^/Wt) , ^nd gjVea also the large fluctuations in 
rate that can res^]!? ^f^^^ \ or 2 Point changes in raw scores, the evaluator 
suggests that obj-^(.^tvA V reaching gain be stated in terms other than 
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Tlable 7 



The Effect on Comprehension Gain Rates of Using Identical vs. Different 
Gatee^MacG-lnltle C-Level Forms, Including a Correction for Apparent 
Difficulty Differences Between the Test Forms 



Variable 
or 

SUtlatlc 


Student group 


C2-C2 

PretestaPrlmary C, Form 2 
Pbsttest^Primary C, Form 2 


C2-<31 

PretestoPrimary C, Form 2 
Posttest-Prioary 0, Form 1 
K«171 


Mean Std* dev. 


Mean Std. dev. 


Pjrctest grade score 


3*1 

0.9 


3.0 ' , 

i.o 


Months enrolled^ 


h.h 

1.8 


2.2 


Posttest grade score 


h.h 

1.1 


Unconnected CI posttest 


h.l 

1.1 


Corrected^ CI posttest 


h.3 

1.2 


Reading gain rate 


3.97 

e.hh 


Uncorrected raio 


2.92 

3.hQ 


Corrected^ rate 


3.62 

3.88 



t test for Independent means and unequal variances 





t based on uncorrected^ rate 


t 


t«2.52 (dfa539, p«.01, 2-tailed) 




t based on corrected^ rate* 




t«0.8 (df«506, p=.U, 2-talled) 



Notes.— Reading gain rate is defined as pre-to-posttest grade score divided by the number of 
months enrolled in the program. 

These gain, rate comparisons are based on the C2-C2 and C2-C1 subgroups at Schools 1, 7, 8, 13, Xh. 
1?,, 17, 19, and 25. 

* "Months enrolled" is the number of enrolled days (including absences) between the pretest and the 
posttest, divided by 17. 9> the average number of days in a school month. 

^To correct for the probable greater difficulty of Cl over C2 tests, 1.5 raw score points were 
added to each Cl test score before calculating grade score and gain rate. 
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monthly grade- equivalent improvement. 

The effect of months enrol l.H |n the Program on reading p ain. Among 
appropriately tested students in both the Intermediate Program and the public- 
school Junior High Program, gain rate was negatively correlated with months 
enrolled in the Program (r was approximately -.30 for each program). 
In general, the longer a student was enrolled in a program, the lower was 
that student's reading gain rate. The negative correlation between gain 
rate and months in the Program also obtained for C2-C1 students (r=-.34), 
suggesting that the remembering passages from pretest to posttestT while a 
possible determinant of reading gain, cannot explain this specific association 
between lower rates and longer enrollment periods. ' 

One plausible explanation for the negative relationship between gain 
rate and enrollment duration is that after a certain duration, further 
attendance in the ESAA class produces diminishing returns. To support this 
explanation, in the four Intermediate Program schools with the highest gain 
rates (Schools 8, 25, 26, and 27), the mean number of months enrolled in 
the Program ranged from 3.2-4.0 (see Table 3). In the three "lower-gain 
schools" (3, 9, 13) the mean enrollments were between 5 and 7 months. In 
the Junior High Program, School F students, who generally met both Objectives 
Sets I and II, had a mean enrollment period nearly one month shorter than 
students in School G, who generally failed to meet their reading objectives. 

The us e of reading materials to supplement the Dorsett and Basic 
Skill lessons . One might have supposed that the use of a variety of read- 
ing materials to supplement Dorsett and Basic Skill lessons would maintain 
student interest and promote reading gain. Among both Intermediate and 
public-scl-ool Junior High Program students, however, gain rate was un- 
correlated with the number of different supplementary materials used. 
Also, "higher-gain" and "lower-gain" Intermediate Program schools were not 
different in the mean number of supplementary materials used by each 
school's stud.ents. In the Junior High Program, the relatively more success- 
ful students of School F each used an average of only one supplementary 
material; the relatively less successful students of School G used an 
•average of 5 different materials. 

As discussed earlier (p. 13), the most common supplement to machine- 
programmed lessons were teacher-made materials. Again, however, the gain 
rates of ESAA students or Schools were not relatfed to the frequency with 
which they used these teacher-made materials. 
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The possibility that teacher differences account for differences in 
reading gain rates . School G was the only school that clearly failed to 
meet both sets of Junior High Program reading objectives during 1974-75. 
During the previous 1973-74 school year, however, School G was among those 
schools with the highest gain rates for Objective I and Objective II 
students (Higgins, 1974c, pp. 22-23). In 1973-74, the School G median 
gain rates for Objective I and Objective II students were 2.8 and 4.5, 
respectively. In 1974-75, the School G corresponding median rates::were 
1.8 and 2.0. For all public school Junior High Program students, the 
median gain rate for Objective I students decreased slightly from about 
2.5 in 1973-74 to 1.7 in 1974-75, whereas the median gain rate for all 
public school Objective II students stayed the same, slightly over 3. 

Between 1973-74 and 1974-75, there was a change in the ESAA teacher 
(and aides) assigned to School G. The 1973-74 teacher was an attractive 
personality who provided a smorgasbord of materials and individualized at- 
tention to students (see Higgins, 1974c, p. 24). A few of last year's 
students, who had completed all their Basic Skill and Dorsett lessons, even 
attempted to "flunk*' their ESAA posttest, hoping to remain in this rewarding 
class. 

The evaluator is not aware of any differences in School G between 
1973-74 and 1974-75 in either the characteristics of the entire school 
or the selection procedure for the ESAA Program. This year's ESAA 
teacher inherited most of the materials prepared by last year's teacher. 
Apparently, these supplementary materials were used in 1974-75, but perhaps 
not with the same effect. The possibility exists, therefore, that differences 
in teacher (and aide) skill and reward value account for some of the ob- 
served differences between School G's 1973-74 and 1974-75 gain rates. 
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RECOMMENDATIONS 



Even though federal and local decisions have already been made to 
discontinue the ESAA Intermediate and Junior High reading programs, the 
evaluator offers the following suggestions in the hope they will be useful 
to those who plan and seek funding for similar programs. 

The first four recommendations should be implemented in the event that 
the ESAA reading programs, or highly similar programs, are funded in the 
future . . 

1- All important materials that Will be used in the reading program 
should be available at the start of the school year . In the Intermediate 
Program, Basic Skill lessons were not placed in the schools until mid-year. 
This delay in the availability of Basic Skill materials seemed to greatly 
hinder their use and usefulness. (See pp. 9-11.) if new reading programs 
such as ESAA are funded, materials should be ordered early enough to be 
on hand when the program starts. 

2. Teachers' desires to have a small budget so that they can select 
their own materials to supplement Dorsett and Basic Skill less ons should be 
considered . Nearly all ESAA teachers indicated that they used supplementary 
materials. And nearly all ESAA teachers indicated that it was at least 
"Moderately important" that they continue to have these supplementary materials 
Furthermore, nearly all teachers felt it at least "Moderately important" 
that they personally select these materials for their own. students. (See p. 48 

3- In the future, teac hers with certification" in reading and/or con - 
siderable training in reading instruction should be recruited for reading 
programs . Only about one-third of the ESAA teachers had reading certifi- 
cation. About one-third of the teachers had taken two or fewer courses 
in reading; among these, four had taken no coursework in reading. (See 
pp. 18-19.) 

^- Staff sho uld carefully screen incoming students to insure that 
students selected are those for whom the reading program is int end ed . 
Although the great majority of 1974-75 ESAA students were well suited to 
the program, 9% of the Junior High Program students had pretest scores 
above the limit specified for program eligibility (see p. 15). 

The following three recommendations are repeated from last year's 
evaluation (Higgins, 1974c, pp. 26-27): 
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5. Althoug h the ESAA reading Programs did a good job of recruiting 
B^lack Americans for staff positions, greater effort should be made to 
recruit Native Americans for any new reading programs . From 1973 to 
1975 no Native Americans were employed as teachers or aides in ESAA 
reading programs. 

6. Supplementary materials (and any newly developed Dorsett and 
Basic Skillis lessons) should be reviewed to insure that they are multi- 
ethnic . Although the objectives of the ESAA reading programs did not 
specify the use of multi-ethnic materials, the adoption of such materials 
whenever possible would serve one purpose of the Emergency School Aid 
Act; namely, the promotion of interracial understanding, 

7. A time-delay feature should be added to the Dorsett machine , so 
that a student who answers incorrectly receives not only a 1-second 
"error tone,'* but also a 10 or 15-second "time-out" period when further 
responding cannot advahce the filmstrip. Two ESAA teachers noted that 
the error tone is not aversive for many students. Such students some- 
times carelessly pressed the machine's buttons until the correct answer 
was located, and so progressed through the lesson without reading it. 

A time delay for incorrect answering would make random button pressing 
much more unpleasant and therefore less frequent. 
The next two recommendations concern testing: 

8. Different, equivalent forms of tests should be used for pre-post 
testing and the computation of reading gains. Data presented on pp. 31-35 
suggest that the use of different vs. identical Gates-MacGinitie C-level 
forms may not affect reading gain rates. Further studies with greater 
experimental control are needed, however, to establish the equivalence of 
gain rates based on alternate vs. same-form pre-post testing. Until such 
controlled studies are undertaken, assessment of^^reading gains should use 
different forms of a given-level test for both pretest and posttest, in 

a balanced design. 

9 . Diagnostic reading tests, appropriate to the age group tested , 

should be used instead of the present Gate s-MacGinitie single-score , 
survey-tyj^e reading tests. For ease of administration and scoring, paper- 
and -pencil diagnostic tests that can be group-admiuistered, should probably 
be used. As discussed on p. 21, most RSAA teachers would like to use a 
diagnostic reading test instead of the present survey test. They have also 
indicated a willingness to spend the extra time needed to score and profile 
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such tests. In the evaluarCT's opinion, a carefully selected diagnostic 
test could provide a meaningful picture of students' reading strengths 
and weaknesses not now available with Gates tests. Such a test could also 
be used to assess reading comprehension gain rates, if required by an 
evaluation design. Teachers might therefore regard the collection of 
diagnostic test scores as useful in their own day-to-day planning and not 
just useful in screening students and summarizing their "reading gains. 

The next recommendation is not a suggestion for improving the ESAA 
reading programs or similar programs, but is instead a suggestion that the 
materials of the 1973-75 ESAA reading programs be preserved for future use: 

10. Appropriate Minneapolis Schools personnel should prepare an 
inventory, of all equipment and materials previously used in the ESAA read - 
j£S>-EE2g£2ES.- A procedure should then be established for (a) distributi on 
of this^ e ^uipm^nt and materials to Minneapolis reading teachers and (h) 
theJTiaintenance of t he items in this inventory, particularly the Dor sett 
machines. 

Based on the ESAA Project Applications for 1973-74 and 1974-75 
(Office of Planning, Development and Federal Programs, Minneapolis Public 
Schools, 1973; and the ESAA Office, Minneapolis Public Schools, 1974), the 
evaluator believes that 214 Dorsett machines were purchased with ESAA funds 
from 1973 to 1975. These machines represent an investment of about $72,500. 
During 1974-75 a total of $5500 was budgeted for maintenance of these machines. 
Now that the OSAA reading programs have ended, the evaluator would suggest 
that these machines (and their accompanying Dorsett and Basic Skill lessons) 
be maintained in proper working order so that they may continue to be a 
valuable resource for Minneapolis Schools. 

The final two recommendations concern future evaluation of experi- 
mental compensatory education programs in Minneapolis such as the ESAA 
reading programs considered in this report. 

11. WTiergyer^pos sible, the evaluation of programs like the ESAA re ady- 
ing 2rogram s should include control groups of students comparable to st ud en t s 
served ^ J: he experimental program . Both this year and last year evaluations 
of the ESAA reading programs were able to state that ESAA students, in 
general, had met program objectives. Knowledge that the ESAA programs met 
their objectives had limited usefulness, in the evaluator 's opinion, for 
two reasons: (a) The objectives may have been chosen arbitrarily, since 
there is no stated rationale for their selection. (b) There is no reference 
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group against whom the gains of the ESAA students can be compared. 

To say that the ESAA programs were successful because students made 
''high" rates of gain would beg the question, "What is a 'high' rate 
of gain?'* Gain rates are dependent on the specific formula used; i.e., 

both on the definition of "time in the program" and, judging from data 
presented in this report, on the particular testing procedure used to measure 
gain. 

The specific formulas used to measure gain rate among ESAA students 
were not used with any other groups of Minneapolis compensatory education 
students. Without control groups administered the same fall and spring 
tests as ESAA students, one cannot confidently assert that the gain 
rates of ESAA students were higher than the gain rates of otherwise com- 
parable Minneapolis compensatory education students either enrolled in 
non-ESAA reading programs or taught reading by their regular classroom 
teacher. ^ 

In short, given the evaluation design used in the present study, onr 
cannot obtain data about the "goodness" of the ESAA programs relative to 
other reading programs.. 

12. Before another evaluation like the ESAA reading evaluation ii 
beguD, program staff sirould r e alistically consider: (a) What, if any , 
are t^he informatiom nggads of program staff? (b) What, if any, are thfc 
information needs ojf the funding agency? (c) What, if any, use will 
be made of results and recommendations arising from the evaluation ? 

In the evaluator's opinion, the mere availability of funds for 
evaluation does not justify conducting an ev-:,Iuat ion. If the local 
program staff and the State or federal funding agency have no important 
information needs that can be served by an evaluator — then no evaluation 
should be contracted. 

The evaluator has no reason to believe that the 1973-74 evaluation 
report was important to the Office of Education's decision to discontinue 
funding the Minneapolis ESAA reading programs. Both during 1973-74 and 
1974-75 the ESAA reading programs generally met their objectives for ' 
comprehension gain among poor readers. 

The 1972 Emergency School Aid Act [Sec. 710. (a) (15)] as interpreted 
in regulations written by Office of Education staff^ requires only that 
ESAA project applications include a research design. The act, as inter- 
preted, does not seem to require that an evaluation based on the proposed 
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design actually be completed; or if completed, that such an evaluation be 
read by Office of Education staff and used as a basis for subsequent years' 
program suggestions and funding. 



Goldberg, Herman R. , Associate Commissioner, Office of Education. Memo 
and proposed regulations concerning applications for i-mergency School 
Aid Act Funds. Washington, D.C., November 17, 1972. 
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Appenidix A 
Hiiane*poU« fiuhlic^5choola 
Stttd«nt^nforajit4cm Foaarfor 197'f-»75 



(1-7) 



rrr 



Stiidfgr'g code ausber 



1 



(8) Blank 

(9) Student's grade placement during 3^4-75. Check one: 

grade 
. 5th grade 
6th grade 

1 Ungraded upper elementary 

■ 2 Other. Please specify: 

(10) Student's sex: 

1 Male 

2 Feaale 



^ Other test. Please specify; 



f ■ I 1 Student's raw score on only the 

(12-13) I I f Coflprehension part of the pretest 

(14-16) [2 



1 P~| Student's grade score on the 
J • I - I Comprehension pretest 



^17-20) I I l/ L I fth. pretest? 

month day 

(21) Vffs the pretest given on the same approxiaate 
date that the student enrolled In the JISAA lab? 
1 Yea 

answer was "No," please explain: 



Posttest Headinp Comprehenaion Test Score Chased on the 
Gfftes^acGinitle test given at the time student left 
the £SAA lab) 

(22) Which posttest did the student take? Check one: 

1 Primary C, Form 2 

2 Survey D, Form 2 

3 Survey D, Form 

^ Other test. Please specify: 



Student's raw score on only the 
Ccmprehension part of the posttest 

Student's grade score on the 



(23-?'») r I I 

. I I j I I Student's grade score 
(25-27) I I I . I I Comprehension posttest 



(?8-31) 



month 



day 



I What was the date 
I the posttest? 



of 



Ataaandance 



Contasttt Pwwon . 

Pawl Higgiru£-;:^;j«:t Evaluator 
Tel. ^kS^ll^:^,tmQ ^ ° 

Minneapolis ®uWic Sch«x>l« 
807 N. Bxnmway 



Between the prcarost and the posttest., and durimr that 
scrxbed forrtthe studemt— ^ 



(33-35) 



T How Qany days vac=±he student present 



(36-38) [ 



Pretest loading Com pi-ehension Test Score (HasW on the 
aates-MacGinitie test given at the time "student enrolled 
in the ESAA remedial reading lab) 

(11) Which pretest did the student take? Check one: 

1 Primary C, Form 2 

2 Survey D, yorm 2 (student wrote on the 

test booklet) 

3 Survey Form (student wrote on a 

separate machine-scorable answer sheet) 



the BSAA renecaaal reading laPT 

How many days wacrthe student absent 
from the lab, evetrrthough the student 
was scheduled to ujce the lab on those 
days? 

(39) Describe the actual schedule by wiui«r,:this stxident 

Check 2nef^ student ^^..«ctu«l attendance) 



that SSkA lab instruction 
the student. 



^irescrlbed for 



(32) Was the posttest given on the sane approximate 
date that the student left the ESAA lab? 
1 Yes 

2 No. If the answer was "No," please explain: 
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_ A Three or four days a week« 
, 3 One or two days a week. , ... 
_ 2 Less than once a week. . 
1 Other schedule. Please describe: 



Reading Ma terials Used by This Studpnt 



On those days when the student was present in the ESAA 

ma-^i^«?"p"/'?.'^: ^^^^ °^ foUowinT 

th ««r^J 1 ^^^^ ^^"^ frequency with which tfee studsnt ufed 

^o*t^ ^r^r % i^.^^'f"*^^'?" appropriate «v»ber in the box 
to the left of the material: 

t ^r"" Z of eirery five cteys attended 

2= Xeaa 1*an^ne:*,y out of fl^«, htRr.dluBais at ieart 

1= iftily once-sar twice while in the lab 
0= never thiBE-year-so far 



ConBierc=::al .linrsett Heading Projrrao 

Minneapalie.Saflic iikillf; Heading Prof:rn« 

Other commercially pret«red m^t±alB rdesigned 
to teach basic Teaaing skills 

Teacher^ade maiEectals to teach h^ic resdinc 
skills ^ 

m) I i 

(45) □ 

(46) □ 

(47) Q 

(48) □ Other materials 

(A9) Did this student ride a bus to school as part of the 
Minneapolis Public Schools' desegregation plan? 
,1 Yes 



High interest books with accompanying cassette 
narration 



Other books or paperbacks (not designed to teach 
reading and ha/ing no reading-related questions) 



Magazines 
Newspapers 



2 No 



(50) 



do not fill in 



For» a 

X2/7h • SB 




Appendix B 
Miimoi^oUa Public Schoola 
Studftfit Infonuitlon Form 
for the 197^75 SSAA Junior Hi^ 



code niaiber 



^ (8) BXwUc 

>49) Student's s^4de placeaent during 197'f-75. Check one: 
. , , 7th grsde 
^ , Sth grade 
9th grade. 

1 Other « pleaae apecify: _. _ 

(10) Student 'a aax; 
1 Kale 

Preteat ^^eadinp CQ|gyhenaioo Test Score (belied on the 
Qatea-4fju:0ij[iitie teat given at the time the student 
enrolled in the fiSAA reading claaa) 

(11) Vhich preteat did the student take? Check one: 
^^^^ 1 Primary Form 2 

. 2 Survey D, Form ? (etudent wrote on the 
teat booklet) 

5 Survey D, Form 2M (etudent wrote on a 
separate aachine^acorable ansver sheet) 

^ Other test* Pl«Me m^^J * 



[m >.| Student's raw score on onuly the 
I CoqKprriienaion part of the pretest 

(1*^16) I [ I 



□ Student* a grade score on. the 
Comprehension pretest 



IWhat was the date of 
the pretest? 



(17-20) I I |./0_ 
month day 

(21) Van the pretest given on the sane approximate 
data that the studcst enrolled in the £SAA lab? 



1 Yes 

? No« If the answer 



"No," please explain: 



Postteat Hasdioff Cqnprehension Teat Score (based on the 
Gate»4fA<;&lx)i tie test given at the time student left 
the SSAA reading class) 

(22) Vhich post test did the student take? Check one: 
1 Primary C, Form 2 

2 Survey D, Form 2 

3 Survey D, Form 2M 

' ' . - ^ Other test* Please specify: _ 



r"^-^ Student's raw score on only the 
Comprehension part of the postteet 

f -"- n Student's grade score on the 



(23-24) [ 

[■ — I- " 1 p~l student's graae score ( 
j • i J Comprehension posttest 

• . month 



day 



What was the date of 
the posttest? 



(52) 1i|aa the poattest given on the same approximate date 
that the student left the KSAA lab? 

1 Yes 

■ 2 No, If the answer was "No/* pleaee explain: 



Attendanca 



Paul Hi 
Tel. 



Minaaapolia Publi^^ 
80? £• Broadv^ ^ 
Minneapolis^ MW h nit^ 



(33-35) 



(36-38) 



Between the pretest and posttest, andyk^ii^y/^ 
was enrolled in the SSAA reading class-^ ^ W 9 

T~1 How many days was the ^^jd^h. 
\ I in the ESAA reading c\^l^ ^ ^ 

] How many daya was the 
\ from the ESAA reading fA^^^l ^ 



I 



(39) Blank 

Reading Materials Used by This Student 



Cta those days when the student was pr«5*^jj^ 1 /^\ ^ 
class I how often did the student use each of 5 ^V/^ 
materials? Rate the frequency with which th^ PS^W^"^^ 
the material, by placing the appropriate nm^ef ^^fc 
to the left of the material: ^ ^ 

5= Svery day, or nearly so 

Three or four days out of every five aj^^ /^^^^^d 
3= C^e or two days out of five • 
2= Less than one day out of five, but (j^^/^^ /t 

several SSAA class sessions ^ 
1= <^ly once or twice while in the ESAft 
0= Never this year so far 



Minneapolis Basic Skills Resding 

Commercial Dorsett Reading Progr^i^ 

Other coonercially prepared matet-^^^^ ^ ^^kt/^^ 
to teach basic reading skills 



Teacher-made materials to teach 
skills 



(wO 

c'»3)n 

(45) 1111 

(46) rn 

(47) [~1 
(46) □ 

(49) Did this student ride a bus to school ^ ij^^ 

Minneapolis Public Schools* desegregatit/^^^k ^ 



High interest books with accompan^^n^ ^ /^tt 
narration 



Other books or paperbscks (not (kf^^^^^ \/^^^^ 
reading and having no reading-rcl^^X^j^ 

Magazines 

Newspapers 

Other materials 



1 Yes 

2 Ho 



(50) 



do not fill in 



52 



St. 



BT 5/75 



Appendix C 
Minneapolis Public Schools 



^ , ■ ■ , — ^ — . ESAA Heading Tteacber Questionnaire 

Do not write In Bpt^cxs above m^'?i i 

, N=31 respondents 

Entries on questionnaire are frequon ci e s 



Parochial 
Parochial 



K^8 
1-8 
9-12 



Your Teaching Situation 

(8) What type of school do you vork in? Check one: 
J^^l An eLementoiy achooi with grades K-3 

■ An elementary school with grades k^6 
12_^ An elooentary school with grades K-6 
■A^^ A Junior hi«b school 1 Parochial 
^ Q A senior bi^b school 3 
- Q_ 6 C>ther. Please specify:!^ 

(9) Are you a (check oue)— 

24 1 PuU-tlme ESAA toAcher? 
6^2 Part-time BSAA tutor? 
(10) d^U^^ru%%^^&%^^r>g naterlala? 
28 1 Yes 
.^ 2 No 

(U) Po you have Basic SkiU Centers Program readinc 
materials? 

21 1 Yes 

9 2 No 1 NA 



Iteturn tos 
Jmil Hlgglns (Ttei. 3W-6ii^) 
Research and Evaluation Dept. 
Minneapolis Public Schooie 
607 V. E« Broadway 
, MiniyapoliS| MW 55I11:} 



ggl^ctlon of fltudenta tar thi> ks aa Readlnt^ Prof ^m 
Please answer below- ^ 

(g2-«5) How was the decision usually rude to enroll a 
' particular student in the ESAA reading proaram? 



(a6-30) What per8on(5) toade this decision In most cases? 



(12) Do you have Dorsett vocabulary lessons? 
14 1 Yes 

11_2 No 1 NA 
On a typical school day. 

(13) How nany hours each dAy do you spend teaching 
reading to ESAA students? Rouad yoar answer to 
the nearest J hour and vrite it in the box below: 

□ (Note: We want the nuaber of 60 minute 
hours. hours, not the number of class periods 
Mean=4.6 exa^c, if you worked Vbours and 

dev. =1.3 • ^^"^^^^^^^^^-^^^^-[S hours., 
(ih) How maiy hours each day do you spend teaching 



(31) Are thei;^' other remedial reading programs in your 
' school (that is, other than the ESAA Reading 

Pi-ogram, or reading instruction by a regular class- 
room teacher who doesn't specUllze in reading)? 

2§_1 Yes 

, 5 2 No 

, j , J I don't know 

1(32-36) If you answered ^'Tes" to (31), please list 
these other remedial reading programs: 



'XCUiS p 

I 3. _b ou 

students other than ESAA students? Hound to Nearest 
i hour: Mean= . 5 

Std. dev.=l .1 



(15) How many hours each day do you spend in school oa 
activities other than teaching students? Round to 
nearest i hour; . Mean=I . 2 

hours. ^. , , ^ , ^ 

^1^^ T*- . dev. =1.0 

Ue>; If you swiervase one or more ESAA Reading Aides, how 



(37-^*1) Given the reading n»terlala you now have, what 
students would you tnost like to have in the 
ESAA Reading Program in your school? Briefly 
describe the reading levels and reading 
problems of those students: 



would you d^^jcribe your working relationship with 
these Aides? Check oner 



20. 

4 



ERIC 



^1 Excellent 
_2 Very good 
J Good 
3 Fair 

J Poor 1 NA 

_6 I do not supervise any ESAA Reading Aides, 

(17) Can you suggest any vaye your relationship with your 
reading aides might be ln5>roved? 
,2 1 yes 

2^2 No 3 NA 

(18-21) If you answered "Yes'* to (17), what are your 
suggestions (please be brief): 



53 

46 



\{h2) In your opinion, la th* ESAA Reading Program with 
I teaching machines particularly weU suited to poor 
readers who are also disruptive and hard to oanaee 
in a reguUr classroom: ^ 

15 _1 Yes 
_S_2 No 

I don't know 1 ^A 
(^3) What proportion of your total ESAA student group 
' came to the ESAA Reading Center Mlnlybac^e thev 
^"^ to manage in nth., .i^, 

and not because they had severe reading problems't — 
Check one: 

_0_i All, or nearly aU, of my ESAA students 

Q 2 More than half... 
,J„3 About half... 

Q. ^ Less than half,.. 
10. ? Very few... 
17^,6 None... 
^ 7 I don't know 



(M»«U8) ?l6s«e Uet the cUvftOi ^ ^^>t^ »<5tlvltl^8 
that your EiSAA atudent^ ti^^y 
to you for ES*A reading '^xr\/^w' 



If you h&Vi Uiftd th« Boriatt ^iweap 

ths f allying qu^gtidSr r^ti^r 

(ii9) Would you favor bulMicg in-^Q Mtchi|j« 



a tUw , delay foar 



the' 



aobther button pxeaa caii f\g^MXi\ 
24 1 Yes 

. 5 NA . 
(50> Do you have afly dleaati^f Ac*;iy(^ ^\ ^\ iHwraet'^ 
machine or its servicing? 
15 1 Yes 
14 2 Ho 
2 NA 

(51-5^) If you answered "Yes" to [xjK^^ 
dissatisfactions : 



(55) If you are familiar vlth Por^-i^^ yJ^\i^X& at 
level A and level AA, woula Zf W C^^H 



one) : ' 

^j^lJ- I«^I A& lessons are g^^^^^ 
tievel A lessons* 

11 2 A lessons are generall:;^ |)^^/^ :5i 
lessons . ^ 

5 3 d and AA lessons have ^ff^\ /^W 
difficulty. 

1 I don't knov« 

-9- 5 I am not familiar wltft t)/^^.-^ tn^^^^rJftls 
at both level A and \e'^^\ ■L 

3 NA 

(56)^ Hbir often do you try to te^c^ ^iiPkt /Hv^a to y^our 
ESiA students? f ^ij^-i^y ^ y 

5^ 1 Once a veek or more 

10 g I^fifl than once a irecH, ri,*^lti/ leaat 
several ESAA class sw^^io^^j 
8 3 Only once or twice thi^ 
8 h Ilever this year so far 

' ■ 47 



^lipencli;^ C (continued) 

Bea^jy^ ^fratg f pr ^gtMeiAa 

(S7) If ve Qii«t meaaure rNiditig^oa|xreheDBlon-gain 
he^ yaar as nart of tBi EEMA evaluation, which 
of the following testa IfdtOd you inpat favor? 
C)»«clc ottiat 



8ama testa aa thia year: the Gates- 
NioOinitie C and D level Comprehension teats 

22 Other level(a) of the Oatea-lHcQinitie 
CoflBpreDenaioii tieata* Please apecify: 



^^0^ Other ctonjnrehenaion tesi by different 

author (tat exaaqple, an appropriate-level 
Stanford Achiervettnt ^st), Haae a teat, 
if you wi^hj ^ . ^ ... .. 



19 k 



A diagnostic reading test (for example, an 
appropriate-level Stanford Diagnostic 
Heading T^at) thai eXaiitf to Bnaitattre (a) 
different ijipea of rMding cotfl^nrehenaion 
and (b) other reodiiis snbskilia* name a 
diagnostic teat, if yot< trishj ^ 



A test other than those apeclfied above. 
Plcaae naioe or deacribe: 



^^5j6 I do not xscnr have a favorite test. 

(58) If Ve uded a diagnostic teat such as the Stanford 
Diagnostic Beading Test next year for the ESAA 
evaluation, would you he willing to score this 
te*t tjot only for CoB^jrehenalon, but also for 5 
or ^ other reading aubskiUfl? (You vould probably 
apand 2 or 3 hours scoring and profiling a set of 
30 terfts.) 

2S^1 Yes 
^3^2 No 

^^^2^ 1 aon't know 

(59) Can yen suggest any other ways that the ESAA test- 
ing procedure might be lirtprovcd next year? 

J^l Yes 

(60-63) If you answered "Yes" to (59), what are your 
stiggestionsT 



ERIC 
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Appendix C (continued) 

Readlnfl Materials Of.her Than Doraett or Basle Skill Centera R^aHing Progr^ma 
For each of the following types of supplementary reading naterlal, please indicate- 
b* ir Z\n2 ^L''"v^" f ^^^''^ students t^is ye^r^ 

" ^^t^??a?s"^SJlrle% ltJlkll'^-.T:rs^l --rial («.™ ^u.ushed 



Reading Material 



Commercially prepared naterlals 
(other than Dorsett or Basic 
Skill Centers programs) desl/?ned 
to teach basic reading skills 



Teacher-made materials to teach 
basic reading skills (these nay 
Include materials you adapted 
Trom cornmercial sources or 
borrowed from other teachers) 



For each reading material listed in the left column, answer both 
Question a and Question b below— ^ 



a. Did you use this type of 
reading material with 
your ESAA students this 
year? 



(6U) 



20 1 Yes 
11 2 No 



High-interest books with 
accompanying cassette narration 



Other books or paperbacks (not 
desii^ned to teach reading and 
having no reading-related 
questions) 



(65) 



24 1 Yes 
7 2 No 



(66) 



21 1 Yes 
102 No 



(67) 



Magazlnas 



Newspapers 



Vocabulary rmte rials (other 
than Dorsett) 



24 1 Yes 
^2 No 



(66) 



_LL1 Yes 
182 No 



(69) 



(70) 



_^1 Yes 
-25.2 No 

1 71 Yes 
142 No 



Games and puzzles 



Other materials 



(71) 



291 Yes 
-2 No 



(72) 



IQ l Yes 

192 No 
2 NA 



Do not write in space 
below: 

(73-79) Blank 
(80) 1 
Card 2 



(1-7) Dup. card 1 



If you answered "Yes" to a, please name or 
describe the material (name published materials 
and describe briefly unpublished naterials).- 



(Q-11) 



(12-15) 



(16-19) 



(20-23) 



(2U-27) 



(28-31) 



(3i^-3'3) 



(36-39) 



{hO-k3) 



48 
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See other side— 



Appendix C (continued) 



Hare Qugatlons on Materials 

(Mf) Kyv loqportant is it tb&t you have extra reading 
' nterials to siqppleneot the Borsett and Basic Sikill 
Centers reading lessons? 

18 1 Extremely iwportwit 

9 2 ItoderatelV important 

2 3 Slightly ingportant 

1 k Hot at all iiH)ortant 

1 NA 

(U5) Hoif important is it that you personally select the 
supplementary materJjxls for your own ESAA students? 

12 I Extremely important 

1^ 2 Moderately important 

2 3 Slightly important 
k Not at all important 



iU6) Did you use any "multi-ethnic" reading mater ials, 
designed to promote understanding of , and respect 
. for, different racial and ethnic groups? 

15 1 Yes 

16 2 No 

(U7-51) If you answered "Yes" to (U6), please describe 
the multi-ethnic rnaterials: 



Your ESAA Training 

You have had several ESAA training sessions, both 
before auid during the school year. Some of these 
training sessions helped you use the Dorsett machines, 
the Dorsett lessons, and the Basic Skill Centers 
lessons. I>u*ing other training sessions you srmrcd 
ideas and problems with other ESAA teachers. 

In general— ' 

(52) How valuable were these ESAA training sessions 
to you? 

13 1 Extremely valuable 
1^ 2 Moderately valxiable 

^ 3 Slightly valuable 

0 k Not at all valuable 
3 NA 

(^3) Can you suggest any ways these training sessions 
might be inqproved? 

14 1 Yes 
16 a No 

1 NA 

(5U-58) If you answered "Yea" to (53), what are your 
suggestions? (Be brief) 



ERLC 



56 
49 



Your General Sugpiestions 

(59) Can you suggest any other ways that the ESAA 

Reading Program in your school might be improved 
next year? 

6 



Jl Yes 

2 



(60-66) If you answered "Yes" to (59), what r^re 
suggestions? 



Personal Data 

(67) What is your highest earned degree? 
21 1 Bachelor's degree 
9 2 Master's degree 
1 3 Other. Please specify: 



(68) Do you ncaw hold tenure as a teacher In the 
Minneapolis Public Schools? 

12 1 Yes 

19 2 No 

(69) Do you now hold Minnesota teacher certifieation 
in reading? 

11 1 Yes 

20 2 No 

(70) If you answered "Yes" to (69), what type(s) of 
reading certificate do you hold (Elementary 
Remedial, Secondary Remedial, etc.)? 



(71-72) In the blank below, write the number of diir.-r*?nt 
college or graduate^school courses in rea^linK 

Mean=4 . 1 
Std. dev. =2.9 



you have taken. 

courses. 



(73) Your sex: 
6 1 Male 
24 2 Female 



1 NA 



(7I+) Your racial/ethnic group. (With which of the 

following racial/ethnic groups do you most closelj' 
identify as a member?) Check one: 

26 1 White American 
5 2 Black American 



0 3 Native American (Indian American, 
First American) 

Q h Spanish-surnamed American (for exampln, 
Chicano-, Puerto Rican, Latin American) 

0 5 Asian American (for example, Chinese, 
Japane s e , Korean ) 

Q 6 Other. Please specify: ^ 



(7*5-79) Blank 
(80) 2 



- RA 5/75 
(1-7) 



Your code number (do not write 
y our ngiae on queatlonnalre) 



Appendix D 

Minneapolii Public Schooli 
ESAA Reading Aide Oueationnaire 

N732 respondents 
Entries on questionnaire 
are frequencies 



Return to; 
Paul Higglna (Tel. 348-6140) 
Research and Evaluation Dept. 
Minneapolia Public Schoola 
807 N.E Broadway 
Minneapolia. MN . 55413 



Tout Work Situation 



(8) 



What type of achool do you work In? 

Check one: (based on all 33 aides) 



An elementary achool with grades 4-6 



0 


1 


7 


2 


9 3 


11 


4 


0 


5 


0 


6 



(9) 



Mean=5. 7 
Std. dev. =1.7 

(10) 



How many hours each typical day do you 
work with pay as an ESAA Reading Aide? 
Round your answer to the neareat 1/2 
hour and write it In the box below: 

I I hours. (For example, if you are paid 
by ESAA for 7% hours per day. 
you would write — 
. hoars . ) 



How would you describe your working re- 
lationahip with the ESAA Reading Teacher 
you assist? Check one: 



24 



0 



0 



^6 Excellent 
__5 Very good 
_4 >ood 
,3 Fair 

2 Poor 

1 Yery poor 



(11) 



Can you suggest any waya to tmprov« your 
relationship with the ESAA Reading Teacher 
you assist? 

2 



30 



_1 Yes 

2 No 



(i2-16) If you answered "Yes*' to (11), what 
are your suggestions? (Please be 
brief,) 



(17) How much do you enjoy your work an an ESAA 
Reading Aide? Check one: 



24 


1 


6 


2 


J 


3 


0 


Jk 


0 


5 



1 NA (not a.scertained) 



(18) 




57 



50 



Are you diaaatisfled in any way with any 
of the conditiona of your eorployment? 
(For example, are you diaaatiafled with 
Civil Service regulationa or pay scales? 
Or do you have any beefa or gripea with 
your employer over working conditiona, 
hours, duties, etc . ) 

Z_l Yes 

25 2 No 



(19-25) If you answered "Yea" to (18), please 
explain your di8Batisfication(s) : 



(26) 



If the ESAA Reading Program were not fundad 
next year, do you think you would have 
trouble finding another Job? 

2_1 Yes 

12 2 No 



1 NA 



17 3 I don't know 

(27-30) If the ESAA Reading Program ia not con- 
tinued, what will you probably do during 
Che next ichool year? 



(31) 



Your Training 

Have you participated in any training ses- 
sions to prepare you for work as an ESAA 
Reading Aide? 

28 1 Yea 

4 2 No 



(32) 



If you answered "Yea" to (31). how valuable 
were these training sessions to you? 

14 1 Extremely valuable 

11 2 Moderately valuable 

3 3 Slightly valuable 

0 4 Not at all valuable 

Can you suggest any ways the training for 
ESAA Reading Aides could be improved? 

10 1 Yes 

22 2 No 



(34^40) If you answered *'Yes*' to (33) what are 
your suggestions? (Please be brief.) 



(33) 



Pleaae anawer queatlona un revt'rse side-*- 



Appendix D (continued) 



Your Job Activltlga 

FImm lilt «h4t yeo comider your tfaiae 
Wmt t^port^it Job ^ctlY^.tUs «■ «n ESAA ftuidios 
A14ft« CxM^Ui of tueli •etivltl«« night Incliide 
. (•) «BCoitr«iiiig or prttisiag oMMits. (b) ko«p» 
'lag i9*to-4«to r»eorda of a^tidmt prognsa, 
(c) orgcnisiag msA atorlng tho reading mtorinla. 
otc, otc. 

(4l«42) nf ttoat important Job activltr is: 



(43«44) Mjr aeeoDd soat iaitortant Job activity is: 



Page 2 

Toor Cetiergl Stt^eatloos 

(63) Can you auggoat any aaya (ocbor than thoae 
yott hava dascribad in ^irioaa ansvera) 
that tha tSAA leading Fcograv in your 
achool ttigjht be iiiprovad next year? 

19 1 Tea 

11 2 No 

2 NA 



(64*73) If you aaavered "Tes"^ to (63). what are 
your auggeationa? 



(45*46) My third w>at isiportafit job activity is: 



(47) Do you now perfora any Job activities that 
you feel you should not perfora? 

1_1 Yes 

^1 2 Wo 

(48-54) If you answered 'nres" to (47), please 
describe these Job activities: 



(55) Are there job activities you don't perfora 

that you feel yow should be performing as an 
ESAA Reading Aide? (Are there thinga you 
don't do that you should be doing?) 

3_1 Yes 



-22,2 Ho 



(56.62) 



If you answered "Yes" to (55), 
describe these job activities: 



please 



Personal Information 

(74) Your sex: 

1 1 Hale 
30 2 Feaale 

1 NA 



(75) 



Your racial/ethnic group. (With which of 
the following racial/ethnic groups do you 
•oat closely identify as a owaber?) 
Check one: 

^6 1 White Aoerican 

4J Black Aserican 

0 3 Native Aoerican (Indian American, 
First Aaerican) 

0 4 Spanish-surnaaed Assrican (for ex- 

aeapla, Chicano, Puerto Rican. Latin 
Aassrican) 

Q 5 Asian Aaerican (for exanple. Chinese, 
Japanese. Korean) 

1 6 Other. Please specify:^ 



EKLC 



SI 



58 



1 NA 

(76-79) Blank 
(80) 1 



